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Abstract. Let H C GLiV) be a connected complex reductive group where V is a finitc- 
dimcnsional complex vector space. Let v G V and let G = {g G GL(V) | gHv = Hv}. Following 
Rai's [Rai'07] we say that the orbit Hv is characteristic for H if the identity component of G is 
H. If H is semisimple, we say that Hv is semi-characteristic for H if the identity component of 
G is an extension of H by a torus. We classify the 7J-orbits which are not (semi)-characteristic 
■ in many cases. 
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Let if be a field. Then H := PGL n (fT) acts on V := M(n,K) via conjugation. There is a 
large literature on solving linear preserver problems, that is, on finding the subgroups of GL(V) 
which preserve a certain set F of if -orbits in V. See [LP01] for a survey. One method of solving 
such problems is to classify all possible subgroups of GL(V) containing if and then check to 
see if these subgroups preserve F. This idea goes back at least to Dynkin [Dyn52] and has 
been used in many papers, e.g., [Gur94, Gur97, GL97, DP93, DL94, PD95]. We generalize the 
problem (but only in characteristic zero) by letting if be a reductive complex algebraic group, 
letting V be an arbitrary finite dimensional representation of if and letting F be an if -orbit 
Hv. The question then becomes: What is the subgroup G of GL(V) which preserves Hvl The 
method of solution is often to look at the possible G and possible G v such that G = HG V 
(which implies that Gv = Hv). We are able to answer the question in many circumstances. We 
are particularly interested in identifying those cases where G° is the image of H, which, in the 
language of Rais [Rai'07], means identifying those ii-orbits which are characteristic. 

Our base field is C, the field of complex numbers. Let V be a finite dimensional ii-module 
where if is a connected reductive group. Let 7^ v G V and set G := {g G GL(V) | gHv = Hv}. 
Then G is a closed algebraic subgroup of GL(V^) (see 2.1 below), We say that Hv is characteristic 
^ ' for H (or simply that v is characteristic for H or just that v is characteristic) if G° is the image 
of H in GL(V). (From now on we will not distinguish H from its image in GL(V), so we will 
say that v is characteristic if G° = H, even though this is not quite correct.) The definition that 
Hv is semi- characteristic is as above, except that we require only that G° is an extension of H 
by a torus (so G has to be reductive). In general, G is not reductive (see Examples 6.12, 6.13, 
7.8 and 7.30). We say that v is almost characteristic if if is a Levi factor of G° and that v is 
almost semi- characteristic if if contains the semisimple part of a Levi factor of G°. 

In §2 we consider some elementary properties of our definitions. We see that one has a chance 
for G° = if only in the case that v G V is generic, which is equivalent to saying that Hv spans 
V. In §3 we consider what can happen to G if we add a trivial factor to V. We show that Hv is 
characteristic if if is a torus and v G V is generic. In §4 we consider the case that if is simple 
of rank at least 2 and V is irreducible. We recall some fundamental results of A. Onishchik 
which apply. We are then able to classify the irreducible if-modules V and v G V such that 
Hv is not semi-characteristic. We determine which orbits are semi-characteristic in the adjoint 
representation of a semisimple group. In §5 we consider the case that if is simple of rank at 
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least 2 and V is reducible. We determine the possible semisimple G containing H such that 
Gv = Hv for v G V. In §6 we consider the case that H is semisimple and V is irreducible. In 
§7 we determine the structure of G when H = SL2. In an appendix we prove branching rules 
which we need to establish our results. 

Our thanks go to M. Rais for his questions and conjectures in [Rai07] which led to this 
paper. We thank Peter Heinzner, Peter Littelmann, Ernest Vinberg and Arkady Onishchik 
for helpful remarks and Alfred Noel and Steven G. Jackson for help with calculations. We 
thank the University of Poitiers, the Ruhr-Universitat Bochum and the Mathematisches Institut, 
Universitat zu Koln for their warm hospitality while this paper was being written. Finally, 
special thanks to the referee for a very meticulous reading of the manuscript and many helpful 
remarks. (S)he also found a serious error in our original version of the section on SL 2 . 



2. Elementary remarks 

We consider when we can remove the prefixes "almost" and "semi." We also reduce to the 
case that Hv spans V. First we show that G is closed in GL(V). 

Lemma 2.1. Let V be a finite- dimensional H-module where H is algebraic. Let G = {g G 
GL(V) I gHv = Hv}. Then G is a closed subgroup o/GL(V). 

Proof. Let d = {g G GL(V) | gHu = Hl>} and G 2 = {g G GL(V) | g{Hv~\Hv) = (Hv\Hv)}. 
Then G\ and G 2 are closed subgroups of GL(V) and G — G± fl G 2 . □ 

We now consider complexifications of compact group actions. Let C be a compact Lie group 
and W a real C-module. Let w G W and assume that Cw spans W. 

Proposition 2.2. Let C, W and w be as above. Let L = {g G GL(Vy) | gCw = Cw}. Then L 
is compact. 

Proof. Fix a basis w±, . . . , w n of W lying in Cw and let 1 1 • 1 1 be a norm on W. Then for g G L 
and 1 < i < n, \\gwi\\ is bounded by a constant which is independent of g. Thus L is a closed 
bounded subset of GL(W), hence compact. □ 

Corollary 2.3. Let H = Cc be the complexification of C acting on V = W ®r C. Let G = 
{g G GL(V) I gHw = Hw}. Then G is the complexification of L, hence reductive. 

Proof. Since Cw is real algebraic [SchOl, Lemma 4.3], it is defined by an ideal / C 1R[W], and 
clearly the complex zeroes of I are Hw. Let I s denote the subspace of / of elements of degree 
at most s, s G N. Then / is generated by some I s . Let fi, ■ ■ ■ , f m be a basis for I s . Then 
g G GL(1U) lies in L if and only if g* fi G I s for all %. This gives a set of real equations defining 
the compact Lie group L, and the complex solutions of these equations are Lq. But the complex 
solutions of the equations are clearly G. Thus G = Lq. □ 

Recall ([Jac62b, Jac79, Ch. II Theorem 11]) that if G C GL(V) acts irreducibly on V, then 
G is reductive. 

Corollary 2.4. Let H be reductive, let V be an H-module and let v G V. Suppose that v is 
almost semi- characteristic for H. Then v is semi- characteristic in the following two cases. 

(1) V is an irreducible representation of H. 

(2) There is a compact Lie group C and real C-module W such that V = W ®k C, v G W 
and H = Cc- 

The following result characterizes when Hv is a cone. 
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Proposition 2.5. Let ^ v G V where V is an H -module. Suppose that Hv is a cone. Then 
there is a 1-parameter subgroup a: C* — >■ H such that v is an eigenvector of a with nonzero 
weight. 

Proof. Since Hv is a cone, v G T v (Hv) and there is an X G f) such that A"(i>) = w. Applying 
an element of H we can assume that X G b, a Borel subalgebra of f). Write X = s + n (Jordan 
decomposition) where s is semisimple and n is nilpotent. Then s and n are in b. We can assume 
that s G t C b where t is the Lie algebra of T, a maximal torus of H . Write v = XIaga Vx as a 
sum of nonzero weight vectors where A is the set of weights of V relative to T such that v\ ^ 0. 
Let $ be the set of positive roots. Then for A G A we have (s + n)v\ = sv\ modulo X^>a 
where \i > A means that n G A + N<&. Thus by an easy induction we get that sv\ = v\ for all 
A G A so that sv = v. Hence S := {t G T \ tv G C*v}° is a subtorus of T which acts nontrivially 
on v. It follows that there is a one-parameter subgroup a: C* — > S as desired. □ 

Proposition 2.6. Lei O^eV where V is an irreducible H-module. Suppose that C*v (f_ Hv. 
Then v is characteristic if it is semi- characteristic. In particular, this holds if v is not in the 
null cone of V . 

Proof. The group G is reductive and its center is contained in the scalar matrices. Under our 
hypotheses on v, the center must be finite. □ 

Let V = ® i=1 niVi be the isotypic decomposition of an //-module where H is nontrivial 
reductive. Let v G V. Then v = (v^) where Vij belongs to the jth copy of Vi, j = 1, . . . ,n i: 
i — 1, . . . , k. Let S denote GL(V) H = GL(m) x ■ ■ • x GL(rifc). Let Ui C K be the linear subspace 
of Vi generated by the j = 1, . . . , rij. If dim Ui = for all i, then we say that v is generic. 

Proposition 2.7. Let H be reductive, let V be an H-module and let v G V . Then the following 
are equivalent. 

(1) The span of Hv is V. 

(2) There is no nontrivial one parameter subgroup of S = GL(V) H which fixes v. 

(3) The vector v is generic. 

Proof. If s G S, then v satisfies one of the conditions if and only if sv does. Clearly, if (1) or 
(3) fails, we can find an s and an i such that (sv)n = 0. If W denotes the first copy of Vi 
in UiVi, we have that C* = GL(W) H C GL(V) H is a one-parameter subgroup fixing v, so (2) 
fails. Conversely, if (2) fails, the fixed point set of a "bad" one-parameter subgroup is a proper 
i/-submodule of V containing v and (1) and (3) fail. □ 

Corollary 2.8. Let v G V and let G be a Levi component of {g G GL(V r ) | gHv = Hv}. Then 
v is generic for H if and only if it is generic for G. 

If v is not generic, then G is in a rather trivial way larger than H. To avoid this case, we 
usually assume from now on that v is generic. 

3. Trivial factors and tori 

Let v G V and suppose that V H = (0). Consider v = (v, 1) G V := V © C. Let G = {g G 
GL(V) | gHv = Hv}. We conjecture that G = G, where G C GL(V) in the canonical way. 
Equivalently, we conjecture that the subgroup of the afline group Aff(V) preserving Hv lies in 
GL(V). Note that v generic implies that v is generic (we can add at most a one-dimensional 
fixed point set). The following example shows that the conjecture fails if H is not reductive. 

Example 3.1. Let H = (C, +) act on C 2 by sending (a, b) G C 2 to (a,ta + b), t G H. Let 
H = H x C where (t, s) ■ (a, b) = (a, ta + s + b), (t, s) G H, (a, b) G C 2 . Then for a ^ 0, the H 
and H orbits of (a, b) are the same, where H C GL(C 2 ), H C Aff (C 2 ) and H £ GL(C 2 ). 
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For this section only G will denote the subgroup of Aff(V) preserving Hv (rather than the 
corresponding subgroup of GL(V)). It is easy to see that we can always reduce to the case that 
V H = (0), which we assume holds for the rest of this section. 

We have a homomorphism Aff (V) — > GL(V) which sends an element (g, c) G G C GL(V) x V 
to g G GL(V). Let G' denote the image of G in GL(V). 

Lemma 3.2. The homomorphism G — >• G' is injective. 

Proof. The kernel K of G — > G' consists of the pure translations in G, i.e., the homomorphisms 
x i — y x ~\~ c where x, c G V. Clearly K is isomorphic to a closed subgroup of the additive group 
(V, +) of V. Now (V, +) has Lie algebra V (trivial bracket) and the exponential map is the 
identity. Thus Ms a vector subspace W of V and K/K° is isomorphic to a finite subgroup of 
(V/W, +). Hence K is connected and K = (W, +) where W must be if -stable. Let it: V — >■ W 
be an if-equivariant projection (here we use that H is reductive). Then there are elements of 
G which translate v to v' where ir(v') is arbitrary. Since H preserves W and Ker7r, this is not 
possible for elements of H, unless W = 0. Hence K is the trivial group. □ 

Note that injectivity fails in the case of Example 3.1. 

Lemma 3.3. Let M be a reductive subgroup of the affine group AS(V). Then there is an 
a G M(V) such that aMa" 1 C GL(V). 

Proof. We use transcendental methods. Choose a hermitian metric on V so that we have a 
unitary group U(V) C GL(V). Let K be a maximal compact subgroup of M. Then M is the 
complexification Kq of K. Now any compact subgroup of Aff(V^) is contained in a maximal 
compact subgroup of AS(V) and all the maximal compact subgroups of AS{V) are conjugate 
[Hoc65, Ch. XV Theorem 3.1]. But clearly U(V) C AS(V) is maximally compact. Thus K is 
conjugate to a subgroup of U(V A ), hence M is conjugate to a subgroup of U(V)c = GL(V r ). □ 

Proposition 3.4. In the following cases G C GL(V). 

(1) The image G' C GL(V) is reductive. 

(2) There is an h' G H such that h'v = Xv, A G C, A ^ 1. 

Proof. If (1) holds, then G is reductive and there is an element a G AS{V) such that aGa" 1 C 
GL(V), hence aHa^ 1 C GL(V). But one easily sees that any affine transformation conjugating 
H into GL(V) must have translation part which is fixed by H. But V contains no nonzero 
if-fixed vectors. Hence G C GL(V). 

Assume (2). Let x H- c + A(x) be an element of g = Lie(G) where ^ c G V and A G flt(V). 
Then the difference of c + A(hv) and c + A(hh'v) is a nonzero multiple of A(hv) and lies in 
\){hv) for any h G H . Thus A itself lies in g and q contains the linear and translation parts of 
its elements. But g cannot contain pure translations, as we saw above. Thus g C flt(V) and 
G C GL(V). □ 

Corollary 3.5. We have that G C GL(V) in the following cases. 

(1) V is an irreducible H-module. 

(2) V is an SLi2-module whose irreducible components are all of even dimension, i.e., a 
module all of whose weights are odd. 

Remark 3.6. Suppose that C, W, w G W are as in Proposition 2.2 where W c = 0. Let L denote 
the subgroup of the real affine group of W stabilizing Cw. Then one can show that L is compact, 
and as above one sees that L C GL(W r ). Complexifying, we see that the subgroup of the affine 
group of V = W ®m C preserving Hw, where H = Cc, is again just the complexification of L, 
a subgroup of GL(V). 
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Proposition 3.7. Let V = (BiJiiVi be the isotypic decomposition of V. Suppose that for no i 
and j do we have that Vi occurs in Hom(Vj, Vi). Then G C GL{V). 

Proof. Suppose that G <f. GL(V). Then we would have a subspace of g consisting of elements 
A w +w, w G W, where W ~ Vi is an irreducible submodule of V, A w G gi(V) and hA w h~ l = Ah w 
for h G if. Our hypotheses imply that A w followed by projection to nAVi is zero. Thus 
exp(yl u; + w)(v) has the same projection to W as v + w. Hence we cannot have Hv = Gv. □ 

Example 3.8. Let V := YH=i m i t fi anc ^ H = n > 1? where ^ is the zth fundamental 
representation of if, i = 1, . . . , n. Then G C GL{V). 

Theorem 3.9. Let H be a torus. Then 

(1) G C GL(V). 

(2) If v EV is generic, then G° = if. 

Proof. We may assume that V H = (0). First consider (1) in the case that if = C*. Let W be 
the subspace of V spanned by if ■ v. Then any g G G must preserve W, so we can replace V by 
W. Thus we can reduce to the case that v G V is generic. This implies that the weight spaces of 
if are one-dimensional. We have a weight basis t> i, . . . , v n of V such that i> = (v\, . . . , v n ) where 
the weight of u$ is 7^ a« G Z. Suppose that the orbit of w is preserved by a transformation 
(g, c) where (g, c)(i>) = . a^fj + q). Here the and q are scalars. Then the zth component 
of g(X ■ v) (where A is a parameter in H = C*) is ^ aij\ aj Vj + q. Now the powers of A that 
occur are distinct, hence the Laurent polynomial in A that gives the zth component has some 
nonzero coefficient for a nonzero power of A. If q 7^ 0, then one can see that the polynomial 
takes on the value for some A 7^ 0. But the C*-orbit of v is nonzero in the ith slot. Thus 
q = for all % and g lies in GL(V) so we have (1). The reasoning above also shows that for 
each % there is a unique j such that 7^ 0. Thus a power g k of g preserves the weight spaces. 
Then g k v = hv for some h G H, and it follows that g k = h. Thus we have (2). Note that g 
normalizes H = C*, so that we actually have g 2 G H. 

Now suppose that if is a torus. As before, to prove (1), we can assume that v is generic. Let 
(g = (ay),c) G G. Choose a 1-parameter subgroup A of H such that all the characters of V, 
restricted to A, are distinct. It follows, as above, that c = and that a power of g lies in H. □ 

3.10. Let Go denote a Levi component of G containing H . Then as we saw before, we must 
have that Go C GL(V). We can write G' as Go x G' u where G' u is the unipotent radical of G' . 
Then we have the corresponding decompostion of q' as $jo ^ d' u - As if -module, q' u is completely 
reducible. Assuming that G is not contained in GL(V) we can choose an irreducible if -module 
W C q' u whose inverse image in q is not contained in gl(V). Then we have a copy of W in V 
and elements A w G Ql(V), w G W, such that x H- A w (x) + w lies in g and {A w } w£ w maps to 
our copy of W in q' u . For all h G if we have hA w h~ l = Ah w . 

Theorem 3.11. Suppose that v G V is generic and in the null cone. Then G C GL(V). 

Proof. Suppose the contrary. Let V = (B^Vi be the isotypic decomposition of V as if -module. 
Let A w + w G Q, w G W be as above where we may assume that W = Vi (first copy) for some %. 
Let 7r: V — > W be an equivariant projection and set = ir(v). Since v is generic, Vi 7^ 0. The 
projection of exp(w + A w )(v) to W has the form Uj + w + p(v, w) where p(v, w) is a polynomial 
which has no linear factors in w and such that the coefficients of the various monomials in w 
are polynomials in v without constant term. By applying elements h G if we can make the 
coefficients of hw in p(hv, hw) as small as we want. But there is no loss if we replace hw by 
w since we are able to consider all possible w. Thus we can assume that the coefficients of the 
monomials in w in p(v, w) are very small, in which case the inverse function theorem tells us 
that w t-j- w+p(v, w) covers a ball around G W whose radius we can choose to be independent 
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of v (for v close to zero). Then we see that w i— > Vi + w + p(v, w) takes on the value 0. Thus 
Gv contains a point which projects to G W, which is impossible. Hence G C GL(V). □ 

Recall that V is called stable if it contains a nonempty Zariski open subset of closed orbits. 

Corollary 3.12. Let V be stable with a one- dimensional quotient. Then G C GL(V). 

Proof. We have that C[V] H = C[/] where / is homogeneous of degree d > 1. Moreover, 
f 1 (f( v )) = Hv = Gv if f(v) ^ 0. Now the case that f(v) = follows from Theorem 3.11 and 
if f(v) 7^ 0, then Gv D Tv where r C G is a finite subgroup isomorphic to Z/<iZ C C* acting 
via scalar multiplication on V. Then G C GL(V) by Proposition 3.4(2). □ 

Remark 3.13. A case by case check shows that H simple and dimV//H = 1 implies that 
G c GL(V). 

Theorem 3.14. If H = SL 2; then G C GL(V). 

We prove the theorem by contradiction, so assume that we have A w + w as in 3.10. Then the 
A w lie in a Lie algebra of nilpotent matrices, and by Engel's theorem we can find a partial flag 
= V C Vi C • • • C V k = V such that V\ is the joint kernel of the A w , V 2 /Vi C V/V\ is the 
joint kernel of the A w , etc. Note that the Vj are if -stable. 

Lemma 3.15. We have W C Vk-i- 

Proof. Since A w {v) G Vk-i-, we have that (A w + w)(v + V^-i) = w + V^-i- Thus exp(yl w , + 
w)(v + Vfc_i) = v + w + Vk-\. Let 7r be the projection of V to V/Vk-i with kernel Vfe_i. If 
W (jL Vfc_i, then ir(Gv) contains a nontrivial if-stable subspace of V/Vk-i- This is not possible 
for the if-orbit, hence W C T4_i. □ 

Lemma 3.16. Suppose that for some j > 1 we have W C Vfc_j and G Vfc_j /or all 

w £ W . Then the stabilizer of v + Vfc_j m H is infinite. 

Proof. Suppose that v + Vk-j has finite stabilizer. Since A w {v) + w projects to zero in V/Vk-j, 
we must have that A w (v) + w = 0, else the G-orbit of ^ has dimension greater than dimH. 
Now for h E H, A w {hv) + w = h(A h -i w + /i _1 w)(v) = 0, so that the average of A w (hv) + w over 
a maximal compact subgroup K of -ff is zero. Since V H = and A w is linear, the average of 
w + A w (/if) over K is to. Thus W = 0, a contradiction. □ 

Lemma 3.17. Suppose that v + \4_j is m the null cone of V/Vk-j for some j > 1. TTien 

Proof. Assume that W <f. Vk-j-x- Our argument in 3.11 shows that the G-orbit of v projected 
to the image of W in V/Vk-j-i contains zero, which is not possible for the if-orbit. Hence 
W C V k - 3 -i. □ 

Lemma 3.18. Suppose that for some j > 1 we have that A w (v) G Vk-j for all w G W and that 
W C Vk-j- Then v + Vk-j is in the null cone of V/Vk-j- 

Proof. Suppose not. Consider V := C-v + Vk-j C V. Then for z G C, z(A w + w) exponentiates 
to an element g(z) of AS{V') which fixes v' := v + Vk-j G V/Vk-j- By Lemma 3.16 we know that 
v' has an infinite stabilizer S in H. Since v' is not in the null cone, S has identity component 
T ~ C*. For any s G S there is an h z ^ s G H such that /i^ jS w = g(z)sv. Then ft, Z)S G S since 
q(z)s fixes u' so that the g(z) preserve the S'-orbit of v. The group generated by T and the 
g(z) is connected, so it preserves the T-orbit of v. By Theorem 3.9 we see that w = 0, a 
contradiction. □ 
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Proof of Theorem 3.14- We have that A w {y) G Vk-\ and W C Vk-\- Suppose that we have 
A w (v) G Vk-j and W C Vk-j for some j > 1. By Lemmas 3.16 and 3.18 and genericity of v we 
may assume that v + \4_j is a sum of highest weight vectors. By Lemma 3.17, W C Vk-j-i, so 
that if A w (v) G Vk-j-i we can continue. We eventually arrive at a case where G" Vfe_j_i 

(we cannot have a pure translation in g). Since v + \4_j is a sum of highest weight vectors 
there are unique elements B w G u such that A w {v) + w + B w {v) G Vfc_y_i. Here u is the Lie 
algebra of the standard unipotent subgroup of iJ. Since A w {v ) + w ^ Vk-j-i, B w {v) G" Vk-j-i 
and w + Vk-j-i is not a sum of highest weight vectors. Since v + Vk-j is a nonzero sum of highest 
weight vectors, the if-isotropy group of v + Vk-j-i, which is a subgroup of the if-isotropy group 
of v + Vk-j, is finite. Arguing as in Lemma 3.16 we obtain that w' := A w (v) + w + B w {v) = 
and that W — 0, a contradiction. Hence G C GL(V). □ 

From now on we will assume that V H = 0, even though we have only established our conjec- 
ture for SL2 or the case that V is irreducible. 

4. The case H is simple and V is irreducible 

Our goal in this section is to find the possible G C GL(V r ) preserving an orbit Hv where V 
is an irreducible if-module and H is simple of rank at least two. We will see that perforce G is 
simple. We begin by recalling some important results of Onishchik. 

Let H C G where G and H are linear algebraic groups. Let V be an if-module. If v G V 
and Gv = Hv, then G = HK where K — G v . Conversely, G = HK implies that Gv = Hv for 
v G V K . There is a rather restricted class of possibilities for H and K when G is simple and H 
is semisimple, as follows from the work of Onishchik [Oni62, Oni69]. 

If K is a connected complex linear algebraic group, let t denote its Lie algebra and let L(K) 
denote a Levi subgroup of K. The next two theorems follow from [Oni62] and [Oni69] (see also 
[Oni94] and [G093]). 

Theorem 4.1. Let H and K be connected algebraic subgroups of the connected reductive group 
G. Then the following are equivalent. 

(1) G = HK. 

(2) G = o~(H)t(K) where o and r are any automorphisms of G. 

(3) G = L{H)L(K). 

(4) Gq = H Kq where H and K are maximal compact subgroups of H and K contained in 
a maximal compact subgroup Gq of G. 

(5) g = f} + £ (if H and K are reductive). 

Corollary 4.2. Suppose that G = HK where all the groups are connected algebraic. Choose 
Levi factors L(G) D L(H), L(K). Then L(G) = L(H)L(K). 

Now assume that f) and £ are reductive subalgebras of the reductive Lie algebra g. Let g s 
be the sum of the simple components of g of rank at least 2 (the strongly semisimple part of 
g). Let G s be the corresponding subgroup of G. Let r(g) be the sum of the center and simple 
components of rank 1 of g so that g = g s © r(g). 

Theorem 4.3. Let f) and £ be reductive subalgebras of the reductive Lie algebra g. Then the 
following are equivalent. 

(1) B = f> + 1 

(2) q s = f) s + t s and r(g) is the sum of the projections of r([)) and r(£) to r(g). 

Corollary 4.4. Suppose that v G V is not semi- characteristic and that H contains a strongly 
semisimple subgroup. Then so does G v . 

From the above and [Oni62] we have the following 
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<Pl + <Pn 


An-1 


3.1 


D 2n 


B 2n -1 


<Pi + 9i 


c 






3.2 


D 2n 


B271-I 




C n X Ai 




C„-i x Ai 


4.1 


B 3 


G 2 


¥2 


B 2 


(fx + 9 2 


Ai 


4.2 


B 3 


G 2 


¥2 


D 3 


<Pi + Oi 


A 2 


5.1 


D 4 


B 3 


V? 3 


B 2 


<Pi + 3 


Ai 


5.2 


D 4 


B 3 


V? 3 


B 2 x Ai 


Vi + Vl 


Ai x Ai 


5.3 


D 4 


B 3 


V? 3 


D 3 


fl + 02 


A 2 


5.4 


D 4 


B 3 


^3 


B 3 


(fii + Q\ 


G 2 


6 


D 8 


B 7 


V^i + 0i 


B 4 


(fi 4 


B 3 



Theorem 4.5. Lei G fre connected, simple and simply connected of rank at least 2. Let H 
and K be connected semisimple subgroups of G such that G = HK. Then, up to switching the 
roles of H and K and replacing each of them by their image under an automorphism of G, all 
possibilities are listed in Table 1. 

In our tables, we always have n > 1 and k > 1. We use 8k to denote a trivial representation 
of dimension k. Corresponding to an ordering of the simple roots of G we have fundamental 
representations (fii = <fii(G), i = 1, . . . ,rankG. We use the ordering of the roots of the simple 
groups of Dynkin [Dyn52]. Note that entries (5.1), (5.2) and (5.3) of Table 1 are special cases 
of (3.1), (3.2) and (2), but we have included them for completeness. 

Corollary 4.6. Let (G,H,K) be a triple in Table 1. 

(1) If L C G is a reductive subgroup commuting with H or K, then L has rank at most 1. 

(2) We have G = H S K S where K s and H s are simple. 

Now that we know the possibilities for G, H and K, our task is to find the irreducible 
representations of G which remain irreducible when restricted to H. This can be read off from 
[Dyn52, Table 5]. However, given that one knows the possibilities for (G,H,K), it is relatively 
easy to see which irreducible representations of G are possible. Note that we can sometimes 
gain an irreducible representation by adding a group of rank 1 to if (Table 2(3.5)). 

Theorem 4.7. Let G = G s be simple and let H and K = K s be proper semisimple subgroups 
of G such that G = HK. Assume that V is an irreducible representation of G which is also 
irreducible when restricted to H. Then, up to automorphisms of G, all possibilities are listed in 
Table 2. 

In Table 2, if V is a i^-module, then S k (V) denotes the i^-subspace of S k (V) generated by 
S k (V u ) where U is a maximal unipotent subgroup of K. In other words, in S k {V) we take 
only the Cartan components of products. In column V K the notation S{f2, fz)k means the 
span of the monomials in f 2 and / 3 of degree k where f\ has degree i, i = 2, 3. Here the 
fi G C[ip 8 (D 8 )} Bi . A similar interpretation applies to S(fi)i- where / 4 G C[(p' r (D 8 )} Bi has degree 
4. The justification of the entries V\K and V K can be found in the Appendix. 

Remark 4.8. In Table 2, we have chosen not to remove all redundancies due to automorphisms 
of groups of type D n . In column V\K we have omitted the decompositions in (3.2) and (3.4) 
which are obtained by restricting V to C n . To determine this one needs the branching rule for 
restrictions of SL 2n -representations to C„. As determined by Weyl [Wey39], one proceeds as 
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Table 2. 





G 


V 


H 


V H 


K 


V K 




1 


A 2rl -1 


h 


Cn 




&2n-2 


<p$ + ^r 1 + --- + e 1 


^1 


2.1 


D 2 n+1 


L 


B 2 n 


I, 


A 2 n 


S k (^l + ( P3 + ■■■ + V2n-1 + #l) 


01 


2.2 


&2n+l 


h 


B2n 


I. 


A 2n 


S k ((f 2 + lf 4 + --- + lf 2 n + 0l) 


01 


3.1 


D 2 n 


h 


B 2 ra-1 


h 

V2n-l 


A 2n -i 


S k (ip 2 + (f4 H h <^2n-2 + 6*2) 


S fc (C 2 ) 


3.2 


D 2 n 


¥>2n-l 


B 2 n-1 


h 

V2n-1 


c„ 


* 


S k (C n+1 ) 


3.3 


D 2 n 


T 


B 2 n-1 


JU 


A 2n _i 


5 fc ((^i + V? 3 H h </?2n-l) 


(0) 


3.4 


D 2 n 




B 2 n-1 


JL 


C n 


* 


(0) 


3.5 


D 2 „ 


<Pl 


C„ X Ai 


¥>1 <S> y?i 


B 2 n-1 


¥>l + 01 


01 


4.1 


B 3 


cpf 


G 2 


<^2 


B 2 


S fe (V?l+0 2 ) 


S k (C 2 ) 


4.2 


B 3 


r 


G 2 




¥?2 


D 3 


«S%1 + 01 ) 


01 


5.1 


D 4 


r 


B 3 


£ 


B 2 


«S%1 + 3 ) 


S fc (C 3 ) 


5.2 


D 4 


r 

n 


B 3 


T- 


D 3 


<S%l+0 2 ) 


^(C 2 ) 


5.3 


D 4 


I, 


B 3 




B 3 


<S%i + 6>i) 


01 


5.4 


D 4 


<P1 


C 2 x A x 




B 3 


y?i + 6>i 


01 


6.1 


D 8 


<P1 


B 4 


v? 4 


B 7 


<Pi + 6i 


01 


6.2 


D 8 


if 7 


B 4 


V?iv? 4 


B 7 




(0) 


6.3 


D 8 


<p k 


B 7 




B 4 


* 




6.4 


D 8 




B 7 


h 


B 4 


* 


S(f2, / 3 )fc 



follows. Let u e A ((C 2n )*) be nonzero and C n -invariant. Then from the exterior powers of 
<jj we obtain invariants in the duals of <p h 2 . . ■^f^n'-i f° r nonnegative bj. Then the restriction 
of an irreducible representation ip of SL 2n to C n is obtained by taking all possible complete 
contractions of the duals of our invariants in the (p^ 2 ■ ■ ■ ^2^-2 with F° r example, <^ 1 (y9 5 (SL 8 ) 
contracted with uj gives rise to y? 4 and <pi<p3 while contraction with 00 A u G A 4 (C 8 )* gives rise 
to if \ and (p2- Note that the only representations of SL 2n which can give rise to the trivial 
representation of C n are those of the form (p^ ■ ■ ■'fitri-i- Hence in Table 2 (3.4) the trivial 
(^-representation does not occur in the column V\K for any k > 1 and in (3.2) the occurrences 
of the trivial representation are the symmetric algebra in 2 and the subspaces (p 2 n , . . . , <p 2 n-2- 

Remark 4.9. We do not know what to put in the column V\K in cases (6.3) and (6.4) of Table 
2. However, in the Appendix we are able to compute V K . 

Suppose that H C G where G and H are semisimple, and connected and V is a G-module 
which is irreducible as an iJ-module. Then the inclusion of H in G has a very special form, as 
shown by Dynkin [Dyn52, Theorem 2.2]. 

Theorem 4.10. Let G\, . . . , Gf. be the simple components of G. Then H = Hi ■ ■ ■ where the 
Hi are nontrivial semisimple subgroups of the Gi, i — 1, . . . , k. 

We are interested in the case that H is simple. Then Theorem 4.10 tells us that G has to be 
simple and from Table 2 we get the following theorem. 

Theorem 4.11. Let H be simple of rank at least two and let (p: H — > GL(V) be an irreducible 
representation. Then every nonzero H -orbit Hv is semi- characteristic, except for the following 
cases (where n > 2 and k > 1). 

(1) H = C n , tp = p\ and v is a highest weight vector. Equivalently, v is fixed by A 2n ,_ 2 where 
A 2n - 2 , C n ,, A 2n „i and V are as in Tables 1(1) and 2(1). 

(2) H = B 2n , <p = ip k n and v is a highest weight vector. Equivalently, v is fixed by A 2n where 
A 2n , B 2n , D 2n+1 and V are as in Tables 1(2) and 2(2.1) or 2(2.2). 
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(3) H = B 2n -i, — f^n-i an d v is fixed by C n where C n , B 2n -i, D 2n and V are as in Tables 
1(3.1) and 2(3.2). 

(4) H = G 2 , (p = (p% and v is fixed by B 2 where B 2 , G 2? B 3 and V are as in Tables 1(4-1) 
and 2(4.1). 

(5) H = B 4; ip = <£> 4 and Hv is closed. Equivalently, v is fixed by B 7 where B 4 , B7, D 8 and 
V are as in Tables 1(6) and 2(6.1). 

(6) H = B 7 , ip = and v is fixed by B 4 where B 4 , B 7 , D 8 and V are as in Tables 1(6) and 
2(6.3) or 2(6.4). 

For special direct sums of representations we have the following result. 

Proposition 4.12. Let Vi be an irreducible Hi-module where the Hi are semisimple, i = 
1, . . . , k. Let V — V\ © ■ ■ • © Vfc be the corresponding H — Hi x ■ ■ • x H^-module. Suppose 
that 7^ Vi G Vi such that Vi is (semi) -characteristic for Hi for each i. Then v is (semi)- 
characteristic for H where v = (vi, . . . , v^) G V . 

Proof. Let G be as usual. First suppose that V is an irreducible G^-module. Then, up to a 
cover and scalar matrices, G° = G\ x • • • x G r where the Gj are simple and V ~ Ui <S> • • • <S> U r 
where the Uj are irreducible Gj-modules. By Theorem 4.10 each simple factor of each Hi must 
project nontrivially to a single Gj. But given the structure of V as if -module, this implies that 
k = 1, where the theorem is trivial. 

We may now assume that there is a maximal flag W\ C • ■ ■ C W r C W r+ i = V of te- 
stable subspaces where r > 1. We may assume that, as if-module, W r = V\ © • • • © V p so that 
V/W r ~ V p+ i@- ■ -®Vk- The image of G in GL(V/W r ) is reductive. Let G' denote its semisimple 
part. Then for g G G', we have g(v p+ i, . . . , v k) G (H p+ i x ■ • • x Hk)(v p+ i, . . . , Vk). By induction 
on k, G' = H p+ i x • ■ ■ x Hk- But by maximality of the flag, we must have that p — k — 1, i.e., 
V/W r ~ Vk- If Vfc is not G-stable, then q contains a nonzero linear map of Vk to W r . Since is 
stable under the action of H, we may assume that it contains Hom(V/ i; , V\). Thus the G-orbit of 
v contains a point (0, w 2 , . . . , Vk)- Such a point is not in Hv, so we have a contradiction. Thus 
Vk is G-stable and we have a G- module direct sum decomposition V = W r © Vk- It follows by 
induction on k that Hv is (semi)-characteristic. □ 

If one considers the adjoint representation (j of a simple H, the only case that appears in 
Theorem 4.11 is H = C n , n > 2, where I) = Thus we have 

Corollary 4.13. Let H = Hi x • • • x fT fc where the Hi are simple, and let tp: H — > GL(f)) be 

the adjoint representation. Let v = (t>i, . . . ,Vk) G ©jfjj where no Vi is zero. Then v is semi- 
characteristic if and only if for every simple component Hi of type C n , n > 2, Vi G c n is not on 
the highest weight orbit. 

5. The case H is simple 

We now consider the case where H is simple of rank at least two and our if-module V may 
be reducible. We consider the possible semisimple G which can act on V such that Gv = Hv 
where v G V is generic for the action of H. 

Here are some examples to keep in mind. 

Example 5.1. Let H be simple and let V be an ii-module. Let G = H x H and let v be the 
identity in V ® V*. Then Gv = Hv where the diagonal copy of H in G plays the role of K. 

Example 5.2. Let (G,H,K) or (G,K,H) be an entry of Table 1. Let V = ©" =1 rriiVi be the 
isotypic decomposition of a C7-module such that dimV^ > rrij > 1 for all i. Let v G V K be 
generic for the action of G. Then Gv = Hv. 
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Example 5.3. Here H ~ A 2n _i. Let (Gi,H 1 ,Ki) = (D 2n , A 2n -i, B 2n -i) and (G 2 ,H 2 ,K 2 ) = 
(A 2n _i, A 2n _ 2 , C n ). Let G = Gi x G 2 , let if denote the diagonal copy of A 2n _ x and let f\~ = 
Ki x f\~ 2 . Then G = HK. Let F be a representation of G which contains a generic vector 
v G V . If V is irreducible, then V — V\ <8> V 2 where Vi is an irreducible representation of 
Gi, i = 1, 2, and f G V^ 1 £g> V^ 2 . The only possibilities allowing nontrivial fixed points are 
Vi = (p\, k > 0, and = <p 2 2 <pl 4 • • • ftri-i where the a 2i are in Z + . In both cases, dim = 1. 
Thus for v to be generic, V must be a sum of representations (each of multiplicity one) of the 
form V\ <g) V 2 where dimV^ i = 1 for all i. For V to be almost faithful the sum must contain a 
nontrivial V\ and a nontrivial V 2 . 

Example 5.4. Here H ~ A 3 . Let (G^H^K^ = (B 3 ,D 3 ,G 2 ) and (G 2 ,H 2 ,K 2 ) = (A 3 ,A 2 ,C 2 ). 
Let G = Gi x G 2 , let if be the diagonal copy of A 3 = D 3 and let K = K x x K 2 . Then G = HK. 
If is an irreducible representation of Gi, i — 1, 2, with Vf^ 1 7^ (0) 7^ V^ 2 , then Vi = yc 3 , 
A; > and V 2 = ip 2 , £ > 0. Again, dimV^ * = 1, i = 1, 2, and the conditions for v generic and 
V almost faithful are as in the case above. 

Example 5.5. Here H ~ B 3 . Let {G U H U K X ) = (D 4 , B 3 , B 3 ) and (G 2 ,H 2 ,K 2 ) = (B 3 , G 2 , B 2 or 
D 3 ). Let G = Gi x G 2 , let H C Hi x G 2 be the diagonal copy of B 3 and let K = K x x f\~ 2 . 
Then G = HK. The possibilities for the Vi having nontrivial AVfixed points are V\ = ip x , k > 
and V 2 = <p x ip 2 if iT 2 = B 2 and V 2 = ip x if K 2 = D 3 where a and 6 are nonnegative. While 
V x 1 has dimension 1, this is not true for V^ 2 , in general, if K 2 = B 2 . Let d e V be generic 
and fT-fixed. Then irreducible G-modules which can occur in V are sums of tensor products of 
modules V\ (g) V 2 where dim > 1, i = 1, 2. 

Example 5.6. Let (Gi, iii, f\\) be an entry of Table 1. Let G = G x x Gi, let if be the diagonal 
copy of Gi and let K = H x x K x C G. Then G = iff\\ It is usually easy to determine the 
almost faithful V x <S> V 2 with fT-fixed points. For example, in the case of (A 2n _i, C n , A 2n _ 2 ), V x 
has to be of the form p^ ■ ■ ■ <p 2 ^S 2 where the a 2 i are nonnegative and V 2 has to be of the form 
<p x or <p 2n _ x for k > 0. On the other hand, if we have (D 8 , B 7 , B 4 ), then V x is of the form (p\, 
k > 0, and we have been unable to pin down exactly which V 2 have B4 fixed points. 

Looking at Table 1 one easily sees the following. 

Proposition 5.7. Suppose that (G,H,K) and (G',H',K') appear in Table 1 where H fl K ~ 
H' C L C G' or H fl K ~ if' C L C G' and L isomorphic to H or K . Then G' is isomorphic 
to L. 

5.8. Let H be simple of rank at least 2 and let V be an almost faithful if-module. Let v G V 
be generic. Let G = G x x • ■ • x G r where the Gi are simple and simply connected and G acts 
almost faithfully on V such that Gv = Hv where H C G. Let A denote a Levi factor of G v . 
Then G = if if. Let pr^: G — >■ Gj denote projection on the ith factor, % = 1, . . . ,r. We may 
assume that prj(if) 7^ {e} for 1 < i < s and prj(if) = {e} for s < i < r where s > 1. Let 
G' = Gi x • • • x G s and G" = G s+ i x • • • x G r . For r > j > s there is a unique simple component 
Kj of K such that pr^i^) = Gj and clearly if" := K s+ \ x • • ■ x covers G". The kernel of 
K" — > G' commutes with H and fixes v. Since iff spans v (Proposition 2.7), the kernel must be 
finite. Hence K" covers its image in G' . Let K' denote the product of the simple components 
of K not in K" . Then K' C G' and the projection of K" to G' centralizes K' . We must have 
that if A'' = G'. 

We may write V = ® Vi ® H^ where the are pairwise nonisomorphic irreducible represen- 
tations of G' and the W\ are representations of G" . Then the projection Vi of f to each Vi £g> Wj 
is a tensor of rank dimWj since v is generic. Let Ui denote the smallest subspace of Vi such 
that Vi G Ui <S> Wi- Then dimf/j = dimWj and Vi G Ui®Wi corresponds to a i<""-equivariant 
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isomorphism of W* onto Ui C V{. In the sense of the following definition, V{ corresponds to a 
subordination a*: (W*,G") -»■ (V U G'). 

Definition 5.9. Let Z, be an Lj-module i = 1,2, where the Lj are reductive. We say that Z\ is 
subordinate to Z 2 if there is a linear injection a : Z\ — > Z 2 and a reductive subgroup L C Li x L 2 
such that a is L-equivariant (for the L-module structures on Zi and Z 2 ). Moreover, we require 
that pr x : L — > Li be a cover. We say that a: Zi — >■ Z 2 is a subordination of Zi to Z 2 . We 
sometimes use the notation a: {Z\,L\) — >■ (Z 2 ,L 2 ) to specify the groups involved. 

We now consider the possibilities for K' . 

Lemma 5.10. Let H , etc. be as in (5.8). Then for 1 < i < s we have pr^K') ^ Gi. 

Proof. Suppose that prj(i\~') = Gi. Then there is a unique simple component Ki of K' such that 
prj(i^j) = Gi. If pij(Ki) = {e} for j ^ i, then Gi acts trivially on Gv, which is not possible. 
Hence pr-(lfi) ^ {e} for some j ^ i, 1 < j < s. Suppose that pr^i^) = Gj. Then no simple 
component of K' other than Ki can project nontrivially to Gi and Gj. Consider the projections 
H' of H and K[ of Ki to Gi x Gj. Then H'K[ = Gi x Gj, and by reason of dimension we must 
have that pr^H') = Gi and pij(H') = Gj. On the level of Lie algebras this says that we have a 
simple Lie algebra q and two subalgebras \)\ and h 2 of g©0 which project isomorphically to each 
q factor such that t)i + f) 2 = Q © 0. But it follows from [Jac62a, Theorem 9] that f)i H f) 2 ^ (0), 
a contradiction. 

We may thus assume that dim pr j(Ki) < dim Gj, hence dim H < dim Gj as well. By Corollary 
4.6 we have that pr^fQ) pij(H) = Gj and that pi j(K') differs from pr^iQ) by at most a factor 
of rank 1. Hence, with H' and K[ as above, we again have H'K[ = Gi x Gj which is not possible 
by reason of dimension. Hence we have that pr^iT') ^ Gi for 1 < i < s. □ 

Corollary 5.11. Suppose that 1 < i < s and that Ki C K' is a simple factor of rank at least 
two such that pr^Ki) ^ {e}. Then pij(Ki) = {e} for i ^ j , 1 < j ' < s. 

Proof. Suppose that pr j(Ki) ^ {e}. If dim if < dimGj and dim if < dimGj , then Corollary 
4.6 shows that pr^iTj) and pr -(if,) differ from pr^iT') and pij(K') by at most groups of rank 
1, so that with H' and K- as above, we have H'K[ = GiX Gj, which is not possible by reason of 
dimension. Thus pr^if) = Gi (or pij(H) = Gj) which forces pij(Ki) = Gj (or prj(i\~j) = Gi), 
contradicting the lemma above. □ 

Theorem 5.12. Let H, s, r, V = 0^ Vi © Wi, etc. be as above. Then one of the following 
occurs. 

(1) s — 1 and H = G\. Then G" = G 2 x • • ■ x G r and v corresponds to a subordination 
(©i W?> G") (0^ V u H) where W* is sent to V*' for each i. 

(2) s — 1, H ^ G\ and G\ = HKi where K\ C G\ is a simple component of K. If r > 1, 
then r = 2, G 2 = SL 2 (up to a cover) and (G\, H, K\ x SL 2 ) is case 3.2 of Table 1. We 
have a subordination (0H / i *,SL 2 ) — > (0VJ,if) where the image of W* is a subset of 
V^ 1 for each i. 

(3) s = 2, pr 1 (if) = G±, pr 2 (H) = G 2 and there are subgroups K[, K 2 of H such that 
(H, K[, K 2 ) occurs in Table 1. We have K = K\ x K 2 where Ki = pr^iT-), i — 1,2 and 
G = HK (Example 5.6). If r > 2, then r = 3, (H,K[,K' 2 x SL 2 ) or (H,K' 2 ,K[ x SL 2 ) 
is entry 3.2 of Table 1, G" = SL 2 and v corresponds to a subordination (®^ W* , SL 2 ) — > 
(0^ Vi, H) where W* has image in v^ lxK2 for all i. 

(4) r = s = 2 where pr 1 (if) ^ G\ and pr 2 (if) = G 2 . Then we are in the case of Example 
5.3, 5.4 or 5.5. 

Proof. The cases where s = 1 are quite easy and we leave them to the reader. Now suppose that 
s > 1. Suppose that pr 1 (if) ^ Gi and that pr 2 (H) ^ G 2 . Then there are strongly semisimple 
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factors Ki of K' such that pr^ifj) pr^H) — Gi, i — 1, 2. By Lemma 5.10 and Corollary 5.11 the 
prj(iTj) are proper subgroups of the G, where px 2 (Ki) = pr 1 (i^ 2 ) = { e }- Applying Proposition 
5.7 we obtain that each of the Gi is isomorphic to H, a contradiction. Thus we may assume 
that pr^if) = G\. 

Let V (resp. Kx) be the product of the strongly simple components of K' which map trivially 
(resp. nontrivially) to Gx- Then L' C G 2 x ■ • • x G s . Since G' = HK' and pr 1 (iJ) = Gx, we 
must have that G2 x ■ ■ • x G s = H'L' where H' is the inverse image of pr 1 (i^i) in H projected 
to G 2 x ■ ■ ■ x G s . Since H' is a proper subgroup of H, pr^H') ^ Gi, i = 2,...,s. Since 
pr 2 (if') pr 2 (Z/) = G 2 , we must have that H' is simple, by Table 1. By our argument above, we 
must have that s — 2. 

Now suppose that pr x (if) = Gi and that pr 2 (if) = G 2 . Then by Lemma 5.10 and Corollary 
5.11 we have that H x H ~ Gi x G 2 = H(Kx x iT 2 ) where the iTj C Gj are images of simple 
subgroups iT{ and iT 2 of K. Then if = K[K' 2 so that (if, fT(, fT 2 ) occurs in Table 1, as claimed. 
Suppose that r > 2. Then for j > 2, Kj C Gx x G 2 x Gj projects onto Gj and commutes 
with and fT 2 . But the centralizer of ifi x K 2 in Gi x G 2 is trivial unless (H, K[, K' 2 ) or 
(if, i<" 2 , is entry 3.1 of Table 1, in which case the centralizer is SL 2 . Thus r = 3 and there 
is a subordination as claimed. 

Finally, suppose that s = 2 and that pr 2 (if) = G 2 and pr 1 (iJ) 7^ Gi. Using Lemma 5.10 
and Corollary 5.11 and the fact that HK' = GiG 2 , we see that there are simple subgroups 
Ki C Gi, i — 1, 2, such that H(KxK 2 ) = GiG 2 . We have entries (Gi, pr : (H), Kx) and 
(pr 2 (if ), pr 2 (L), K 2 ) in Table 1 where L is the preimage in H of pr 1 (if) fl A'i. Then H must be 
B 3 or of type A 2n _i. If H = B 3 , then one easily sees that we are in Example 5.5 and that we 
cannot have r > 2. The remaining possibilities are that H = A 2n -i and G\ = D 2n or B3 giving 
Examples 5.3 and 5.4 where r = 2 is forced. □ 

The theorem above gives one the possibilities for the semisimple part of the Levi factor of 
{g G GL(V) I Gv = Hv}. Preferable would be a theorem which starts with a representation V 
of H and a generic v £ V and tells you when v is almost semi-characteristic. In general, it is 
rather cumbersome to give such a theorem (for SL 2 see section 7). We content ourselves with 
working out the following example. 

Example 5.13. Let H = D 2n+ i, n > 2. Let V = ©^ =1 ^iUi be the isotypic decomposition of the 
if-module V. Let v = (vx, ■ ■ ■ ,Vk) G V be generic. We find conditions which guarantee that v 
is almost semi-characteristic. 

Each Vi is {vix, ■ ■ ■ ,Vi,m) where Vij lies in the jth copy of Vi, and the Vij span a subspace 
Ui C Vi of dimension rij. In order to avoid case (1) of Theorem 5.12 we have to assume that 
the intersection of the stabilizers of the subspaces Ui in H contains no nontrivial semisimple 
group. Cases (2) and (4) do not apply, so we are left with case (3), where we have G = H x H , 
Kx = B 2n and K 2 = A 2n . But then there is a copy of A 2n _i in D 2n+1 which fixes our point. We 
have already ruled this out. 

6. Semisimple groups 

We turn our attention to the case that H C G where G and H are connected semisimple, V is 
an irreducible if -module, G acts almost faithfully on V and Gv = Hv for some nonzero v G V. 
Let Gx, ■ ■ ■ , Gfc be the simple components of G. Then Theorem 4.10 tells us that H = Hx ■ ■ ■ H^ 
where the Hi are semisimple and lie in Gi, % = 1, . . . , k. Note that no Hi is trivial, else Gj acts 
trivially on V. Thus if Gi has rank 1, then Gj = Hi. We have V = Vx ® ■ ■ • <S> where Vi is an 
almost faithful irreducible representation of both G, and Hi, i = 1, . . . , k. 

6.1. Suppose that G = HK where K is semisimple and G, H and V are as above. (Think of 
K C G v .) Let pr^ denote the projection of G to Gi, i = 1, . . . , k. Let K' be a simple component 
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of K and set I' := {i \ Hipi^K') = Gi and Hi ^ G{\. We may assume that K contains no 
simple component of rank 1 . 

Proposition 6.2. Let G = HK as above. Let K' , K" be distinct simple components of K and 
let I' and I" be as above. Then 

(1) rm" = 0. 

(2) |/'| < 2. If |/'| = 2, then pr^K') = G, for some i G V . 

Proof. For any % G V fl I", the (nontrivial) images of K' and K" in Gi commute. This is 
clearly not possible if pi i(K') is Gi. If not, then we are in one of the entries of Table 1, and 
commutativity is not possible if pr^i^') is one of the groups occurring there. Hence (1) holds. 
Suppose that i, j G I', i ^ j and pr i(K f ) ^ Gi and pij(K') ^ Gj. Then we must have that 
HjL = Gj where L = pr J (pr~ 1 (ifj) fl K') is a proper subgroup of pTj(K') as in the last column 
of Table 1. But then, by inspection, we cannot have HjL = Gj. If i, j and k are distinct 
elements of then we can assume that pr^i^') = Gi, and we derive a contradiction as before 
by considering the non-surjective projections of pr^ 1 (Hi) fl K' to Gj and Gk- Thus we have 
(2). □ 

Theorem 6.3. Suppose that k = 2 and that Hi ^ G\, H2 7^ G2 and Gv = Hv for a nonzero 
v G V. Then we are in one of the following cases. 

(1) There are tuples (Gi, V h Hi, K t ) in Table 2, % = 1, 2, and v e V* 1 ® V 2 Ki . 

(2) The tuple (G U V U H U K X ) is entry (1) of Table 2, (G 2 , V 2 , H 2 , K 2 ) is entry (3.3) (with 
the same k) and v generates the one- dimensional space of h 2n -i fixed vectors in V\ ® 
(V 2 \A 2n ^). 

(3) The tuple (d, Vx, H lt Kx) is entry (6.2) of Table 2, (G 2 , V 2 , H 2 , K 2 ) is entry (6.3) (with 
k = 1 ) and v generates the one-dimensional space of D 8 fixed points in V\ ® V 2 . 

Proof. Let K\ denote a maximal strongly semisimple subgroup of G v . Suppose that ifi C Gj. 
Then Table 1 implies that H\K\ = G\. Let K 2 be a strongly semisimple subgroup of G v such 
that pr 2 (K 2 )H 2 = G 2 . Then we must have pr 1 (i^ 2 ) = {e} (again by Table 1), hence K 2 C G 2 
and we are in case (1). Thus we may suppose that any maximal strongly semisimple subgroup 
L of G v lies diagonally in Gi x G 2 . Since we are not in case (1), pr 2 restricted to L is almost 
faithful (and so is pr x ). By Table 1, L must be simple. It follows from Proposition 6.2 that we 
have two cases: 

Case 1: pr 1 (L) = G\ and pr 2 (£) = K 2 where (G 2 , H 2 , K 2 ) is in Table 1. Moreover, (G 2 , H 2 , K' 2 ) 
is in Table 1, where K' 2 = pr 2 (prj" 1 (iJ 1 ) fl L). Thus we are in the case (G 2 , H 2 , K 2 ) = 
(D 2n , B 2n -i, A 2n _i) and (G 2 ,H 2 ,K 2 ) = (D 2n , B 2n _i, C n ), n > 2, where H x ~ C n . Then from 
Table 2 we see that V\ ~ <^i(A 2n -i) (or its dual). From Table 2(3.3) we get possibility (2) of our 
theorem. From (3.1) we get nothing since A 2n -i has no fixed vectors in V\®V 2 . The possibilities 
(4.2) and (5.2) fail for the same reason. Hence we only get (2). 

Case 2: Here we have that pr^L) = G\ and pr 2 (L) = G 2 . Then L = H[H 2 where H[ = 
pr^ 1 (Hi) nL, i — 1, 2. Moreover, there are irreducible representations Vi of L whose restrictions 
to H[ are irreducible, 2 = 1,2. Table 2 tells us that we may have possibilities from entry (5.3) 
(and isomorphic entries), but then there are no D 4 -fixed points in V\ ® V 2 . Finally, from (6.2) 
and (6.3) we get possibility (3) above. □ 

6.4. Let H C GL(V^) where H is semisimple connected and V is an irreducible if -module. 
Suppose that v G V is a nonzero orbit such that the connected semisimple part G of {g G 
GL(V) I gHv = Hv} is strictly larger than H. Let K denote the strongly semisimple part of 
G v . We are then in the situation of 6.1. For each simple component Kj of K, let Ij C {1, . . . , k} 
be as in 6.1. Set /' = Ujlj, V = ®i^pVi and V" = ®i0*Vi. Define G' and G" analogously. 
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Then G" = H" = lUfii' Hi. Let K' be the product of the Kj such that Kj C G' and let K" be 
the product of the other simple factors of K so we have K = K'K" . Via the projections to G' 
and H" we have i^'-module structures on V and V" . Let W C V be the minimal subspace 
such that veW'& V" . Then W C (V') K ' and v is £T"-fixed. 

Remark 6.5. It follows from Theorem 6.3 that each simple component of K' arises from an entry 
of Table 2 as in Theorem 6.3(1), is a group f\ 2n -i as in Theorem 6.3(2) or is the group Dg in 
Theorem 6.3(3). 

We restate the discussion in (6.4) as follows. 

Theorem 6.6. Let v G V. If Hv is not semi- characteristic, then there are K' , K" , etc. as in 
(6.4) and a minimal K" -stable subspace W C (V') K ' such that v G W ® V" . If K" ^ {e}, then 
there is a subordination a: ((W')*,K") — > (V",H"). 

Now we would like to find some simple sufficient criteria for all generic v to be semi- 
characteristic. For this, we only need to avoid the case that Gv = Hv where Gi differs from 
Hi for only one i. Then after renumbering we have that G\ ^ Hi and Gi = Hi for i > 1. We 
have that V" = Vi ® ■ ■ ■ ®Vk and H" = Hi X • • • x Hk- Note that H\ may be any semisimple 
subgroup of H . 

Proposition 6.7. Let G\ D H\ be as above. Then one of the following occurs. 

(1) There is a subordination a: (V*,Gi) —> (V",H") where K" projects onto G\. 

(2) The tuple (Vi,Gi, H±, K\) occurs in Table 2 where K\ is the projection of K" to G±, 
and we have a subordination (W*,Ki) — > (V",H") where W\ is minimal such that 
v e Wx®V". 

(3) The group K" projects trivially to G\ and the tuple (Vi, Gi, Hi, K\) occurs in Table 2 
for some K\ where V^ 1 7^ (0) . 

Proof. If pr 1 (J^") = {e}, then we are in case (3). Suppose that pr 1 (K") is nontrivial. Then it 
follows from Table 1 that K' = {e}. If the projection of K" to Gi is G±, then v corresponds 
to a subordination of V* to V" and we are in case (1). The only other possibility is that the 
projection of K" is K\ where (Vi, G\, H\, K\) occurs in Table 2 and we are in case (2). □ 

Example 6.8. Suppose that k = 2, dimV2 > dimVi and that H 2 = SL(V2). Let v G V\ ® Vi 
have maximal rank. Then case (1) applies. If (V\,G\,H\,K\) occurs in Table 2, let W\ be any 
nontrivial i^i-subspace of V\ and let v G W\ ® Vi have maximal rank. Then case (2) applies. 

From Proposition 6.7 we get the following criterion for all nonzero orbits Hv to be semi- 
characteristic. 

Corollary 6.9. Let V be an irreducible H-module where H is semisimple. Write H = H\ x 
• • • x Hk where the Hi are simple for % > 1, and let V = V\ <g> • • • <g) V^ be the corresponding 
decomposition ofV. Let V" denote Vi <g> • • • <g> Vf. and set H" = H 2 ■ ■ ■ Hk- Suppose that none of 
the following occurs for any decomposition H = Hi x • • • x Hk. 

(1) There is a subordination (V{,H{) -> (V", H") where H x ^ SL(V[). 

(2) There is a tuple (Vi, G x , H x , K x ) in Table 2 and a subordination (W*,K X ) -> (V",H") 
where W C V\ is K x - stable. 

(3) There is a tuple (V 1 ,G 1 ,H l ,K 1 ) in Table 2 where V x Kx ^ (0). 

Then every nonzero v G V is semi- characteristic. 

Admittedly, the corollary is a little unwieldy, but in any concrete case it is quite easy to 
apply. We see what we can say in the case of isotropy representations of symmetric spaces. 
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Example 6.10. Let H = A5 x Ai acting on V — tps © <Pi- This corresponds to the symmetric 
space of type EII (see [Hel78, Ch. X, Table V]). Let 7^ v G V and let G be as usual with 
semisimple part G s . If G s contains H, then G s cannot be simple (by Table 2), and if it is of the 
form G\ x G 2 where G\ D A5 and Gi = SL2, then it follows from Corollary 6.9 or Proposition 
6.7 that G\ = A5. Hence v is semi-characteristic. 

Example 6.11. Let p > q G N where p > 1. Let H = Sp 2p x Sp 2? act in the natural way on 
V = C 2p © C 2q . This corresponds to the symmetric space of type CII. Let 7^ v G V. Then 
one easily sees that the only possibility for a semisimple G s containing H stabilizing Hv occurs 
in the case that v has rank 1, in which case G s = SL 2p x SL 2q . For q > 1 this corresponds to 
Theorem 6.3(1). If rank v > 1, then v is semi-characteristic. 

Example 6.12. Let p > q > 1. Let be the intersection of the block diagonal copy of GL P x GL f/ 
in GL p+q with SL p+(? . Then if acts naturally on V © V* where V = Hom(C p ,C 9 ). This is an 
isotropy representation corresponding to the symmetric space of type AIII. First suppose that 
q > 2. Let v = (x,x*) where x G V and x* G V* are nonzero. Let G = {g G GL(V © V*) \ 
gHv = Hv}°. Suppose that v is not semi-characteristic. Then g is an if -stable Lie subalgebra 
of Eom(V®V*, V@V*) which properly contains h. If q projected to Hom(y, V) or Hom(y*, V*) 
is more than a central extension of (), then g has to contain sl p+q . But there is no corresponding 
entry in Table 2. Thus we can suppose that $j projects nontrivially to one of the irreducible 
components of 

Hom(V, V*) ~V*®V* ~ (S 2 (C P ) + A 2 (C P )) © (S 2 ((C q )*) + A 2 ((C)*))). 
Let us consider the case that g contains g' := A 2 (C P ) © A 2 ((C 9 )*). Now x has normal form 
Yli=i e i ® /j where e\, . . . , e p is a basis of C p , /1, . . . , f q is a basis of C 9 and e*, . . . , e* /*,...,/„ 
denote the elements of the dual bases. Then ei A e 2 © /j* A / 2 lies in g' and applied to x gives 
us = ei © /j* + e 2 © /| if > 2. The contraction of x and y* (an if-invariant of V © V^*) is 
not zero. Thus (x,x* +y*) cannot be in the if-orbit of v, a contradiction. If k = 1, then acting 
by the reductive part of H x we can bring x* to the normal form 

1 

cei © /* + ei © /* + e © j{ + ^ ei © /* 

i=2 

where c G C, /* G spanj/l, . . . , /*} and e G span{e 2 , . . . , e p }. If c 7^ 0, then acting by unipotent 
elements of H x we can arrange that e and /* are zero. Then I + 1 is an invariant of x* (its 
rank) under the action of H x . But we can change I by adding elements of g' applied to x, again 
giving a contradiction. If c = and q > 2, then one similarly sees that we can change the rank 
of x*. If c = 0, q = 2 and e or /* 7^ 0, however, the G'-orbit of v is contained in the if x -orbit of 
u and v is not semi-characteristic. The other three possible components of g give nothing new. 
Thus v is possibly not semi-characteristic only when q = 2, v is in the null cone and one of x 
and x* has rank 1. 

If p = 1 and g = 1 we have a torus action in which case G = H . If q = 1 and p > 2 we have 
the action of GL P on C p © (C p )*. If Hv is not closed, then v is semi-characteristic. If iff is 
closed, then G ~ S0 2p and v is not semi-characteristic. 

Example 6.13. In general, one has a good chance to have points v which are not semi-characteristic 
in case your representation is reducible. One can calculate that this actually occurs for the fol- 
lowing isotropy representations of symmetric spaces. 

(1) (V, H) = {C p © C 2 , SO p x S0 2 ), p > 3. This is of type BDI and of type CI for p = 3. 

(2) (V, H) = (A 2 (C n ) © A 2 ((C n )*), GL n ), 3 < n < 5. This is of type DHL 

(3) (V,H) = (<f4 © v\ + y?5 © z/_i, D 5 x C*). This is of type EIII. Here Vj denotes the 
one-dimensional representation of C* of weight j. 
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(4) (V, H) = ((fx © v x + <^ 5 © v-i, E 6 x C*). This is of type EVIL 

Our discussion above establishes 

Proposition 6.14. Let (V,H) be the isotropy representation of an irreducible symmetric space. 
Then, with the exception of the adjoint representation of C n , n > 2 and the exceptions noted in 
Examples 6.11, 6.12 and 6.13, every orbit Hv, v generic, is semi- characteristic. 

The proposition applies to some of the questions of Rais in [Rai'07]. 

7. Representations of SL 2 

We consider the case of if-modules V where H := SL2 and V H = 0. We have a generic v G V 
and G := {g G GL(V) | gHv = Hv}° is not equal to H. We denote by R n the if-module of 
binary forms of degree n. Then R n ~ S^C 2 ) has basis x n , x n ~ 1 y, . . . , y n where x, y are the 
usual basis of C 2 and x n is a highest weight vector. Let Nq{H) denote the connected normalizer 
of H in G. 

To determine G, we show that it suffices to determine N G (H) and g u . We determine N G (H) in 
Theorem 7.4 below. We show that g u is abelian and a multiplicity free if-module (Proposition 
7.15). We give necessary and sufficient conditions for g u to contain a copy of R p , p > 
(Theorem 7.27). We then find some simple conditions that guarantee that g u is zero or the 
trivial if- module for every generic v G V (Corollary 7.29). 

Lemma 7.1. Let G be a Levi component of G containing H. Then G C Ng{H). 

Proof. It follows from Theorem 4.3 that G v contains the simple components of G of rank at 
least 2. These components are normalized by H so they fix the whole orbit Hv which spans V. 
Thus all the components of G have rank at most 1 and G C N G (H). □ 

Corollary 7.2. We have g ~ g x g u . Hence G 7^ N G (H) if and only if g u , as H-module, 
contains R p for some p > 0. To determine G it suffices to determine Ng{H) and g u . 

We now consider the possibilities for Ng(H). 

Example 7.3. (1) Let H be another copy of SL 2 and let Vk — Rjc <S> Rk, k > 1, where 

is the .ff-module of binary forms of degree k.. Let Vf. G Vk be a nonzero fixed vector 
of {(h, h)} C H X H. Then Hvk = (H x H)vk and Vk is generic. We can also take 
V = Vfe © • • • © Vk e and v = (v^, . . . , Vk e ) where k\ < • ■ ■ < kg. Then v is generic and 
N G (H) D HH. 

(2) Let V = Q) k€F rrikRk where 1 < mj, < k + 1 for all k and F is a nonempty finite subset 
of N. Let B and B be the standard Borel subgroups of H and H, respectively. Let Vf. 
be the highest weight vector of the copy of Rik+2-im k in Vk = Rk © Rk for the diagonal 
if-action. Then Vk lies in Rk tensored with the span of the weight vectors of Rk of 
weight at least k — 2m& + 2. Now Vk is an eigenvector for the diagonal copy of B, with 
weight 2k + 2 — Irxik- For b E B, let x(6) denote its upper left hand entry. Let b act 
on R k as the tensor product of the usual action and the scalar action b H- ^(5) 2m *- 2fe - 2 . 
Then Vk is fixed by the diagonal in B x B. Assume that > 2 for some so that 5 
acts effectively. Set v = @ keF Vk- Then v is generic and N G (H) D 

(3) Let = © fceF iTikRk as above. Let w G V be a generic vector whose projection Vkj to 
the jth copy of Rk is a weight vector. If the weight is not zero, then there is an obvious 
C*-action on this copy of Rk such that Vkj is fixed by the product of the standard torus 
in H and our external copy of C*. If v is not a sum of zero weight vectors we have 
N G (H) D HC*. 
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Theorem 7.4. Let V = Q) k€F m k R k be a representation of H = SL 2 where V = 0. Let v G V 
be generic. If Nq(H) 7^ H, then, up to the action ofY[ ke pGL mk , we are in one of the cases of 
Example 7.3. If G D HH as in Example 7.3(1), then G = HH. 

Proof. We have Ng(H) = HG' where G' is the identity component of the centralizer of H in G. 
The group G' v fixes Hv, so it is trivial. Hence the Lie algebra of G v = {hg' \ hg'v = v} projects 
onto g' and into f), so that G' is locally isomorphic to a quotient of a connected subgroup of H. 
Hence G' is locally isomorphic to a connected subgroup of H . 

Case 1: G' = SL2 or SO3. Going to a cover, we can assume that G' = H so that G v is isomorphic 
to the diagonal copy of H. Then V is a sum of representations V k = R k ® S k where S k is a 
representation of H of dimension at most k + 1 and the projection of v to V k is a fixed point of 
the diagonal action of H. Thus S k — Rk and v is as in Example 7.3(1). Suppose that g u 7^ 0. 
Then, as (H x ff)-module, g u cannot contain R or Rq since the connected centralizer of H is 
H and vice versa. Thus g u contains a term R a ® Rf, where ah 7^ 0. Hence, as if-module, g u is 
not multiplicity free. But this contradicts Proposition 7.15 below. Hence g u = and G = HH. 
Case 2: G' = C*. Then G v C H x C* is a diagonal torus and the fixed subspace of G t , on each 
isotypic component m k Rh of V is a sum of m k distinct weight spaces of H. Thus we are in 
Example 7.3(3). 

Case 3: G 1 D U and G' ~~t> H where U C H is the standard maximal unipotent subgroup of our 
second copy H of SL 2 . We have V = © fcgF -Rfc ® ^fe where S k is a representation of U. The 
isotropy group of v in if x U can be taken to be the diagonal copy of U in U x [7. Then w 
corresponds to a subordination SI ^ Rk, hence the image of is a [/-stable subspace of R k 
and Sfc is a 5-stable subspace of R k - In fact, it is the span of x k , x k ~ l y, . . . , x k ~ rnk+1 y mk ~ 1 , and 
acting by elements of the various GL(m k ) we can arrange that v is as in Example 7.3(2). Hence 
N G (H) ~ ffS. □ 

Corollary 7.5. Lei 5e as above and v £ V generic. Suppose that Example 7.3(1) does not 
apply so that Nq(H) ^ HH. Then v is almost semi- characteristic. 

We now turn to the determination of q u when it is not zero or a trivial if-module. 

Proposition 7.6. Let v £ V be generic. Suppose that there is a copy of R p in gl(V), p > 0, 
which is a Lie subalgebra and acts nilpotently on V. Further suppose that R p (v) C f)(f). Then 
Rp Qu ■ 

Proof. Consider the action a: R p ® V — > V . Then for h e H, 

a(R p <g> h(v)) = ha(R p ® v) C hi)(v) = \){hv). 

Hence Hv is open in G p v where G p is the connected group with Lie algebra f) k i? p . Thus 
G p C G and R p C g. The projection of Rp to Lie(Nc(H)) is trivial (by our classification of 
Ng{H) and the fact that R p is nilpotent). Hence R p C g u . □ 

Remark 7.7. The proposition remains true if p = as long as G ^ if.£f as in Example 7.3(1). 

Example 7.8. Let p, I > 0. Let = x l+p and let ^ = a^+aix^y, a\ 7^ 0. Set V = Ri +P +Ri. 
Consider a nonzero equivariant map a: R p ® Ri+ P —> Ri- Then a(x 1 y p ~ t <S> vi+p) vanishes for 
i > and o~{y p ®vi +p ) is a nonzero multiple of x . Thus we may arrange that a(y p &)x l+p ) = a±x . 
If A G sl 2 is xd/dy, then <t(?/ p ® ^i+p) = ^4(^z)- We may consider a as an equivariant mapping 
of Rp to Hom(_R; +p , R{). Then i? p applied to v := vi +p + vi is the same as u(v) where u = C ■ A. 
By Proposition 7.6, R p C g u . We can also have a copy of R q in g„, 7^ p, by adding to V 
and adding to v, where v i+q = x l+q . 

We now try to pin down the structure of V and v The situation can be quite complicated. 
First we need a lemma. 



LINEAR MAPS PRESERVING ORBITS 



19 



Lemma 7.9. Let <p: R p <g> R n — > R p+n _ 2 i be equivariant and nonzero where < i < min-jjo, n}. 
Then p{x p ~^y^ ® x n ) ^ for i < j < p. 

Proof. If ip(x p ~^y^ (g) x n ) = 0, then the s[ 2 -submodule W of R p ® i? n generated by x p ~^y^ (g) x n 
lies in the kernel of tp. Applying xd/dy G sl 2 repeatedly we may reduce to the case that 
ip(x p ~ l y l ® x n ) = 0. Suppose by induction that x p ~ k y k ® x n ~ l y l lies in W for k + l = i and / < s. 
Then applying yd/dx followed by xd/dy to x p ~ k y k ®x n ~ s y s we obtain elements in W as well as 
k(n — s)x p ~ k+1 y k ~ 1 ® x n_s_1 y s+1 . Thus W contains all the weight vectors of R p <S> R n of weight 
p + n — 2i. This implies that R p+n _ 2 i C W, a contradiction. Thus ^>{x p ~iyi ® x n ) ^ 0. □ 

Remark 7.10. By reversing the roles of x and y one has that ip{x^y p ~^ ® y n ) ^ for % < j < p. 

Corollary 7.11. Let (p, etc. be as above where p + n — 2i ^ 0. Let w = x n G R n . Then 
dim(f(R p ® w) > 2 unless i = p < n so that <p(y p (&w) is a highest weight vector of R n - P - 

Set Wo = V and for j > set Wj = g u (Wj-i). Then W,- is a proper if-stable subspace of 
Wj_i for j > 0. Let be the greatest integer j such that Wj ^ 0. Since U acts nontrivially 
on V, we must have k > 0. Let be an .//-complement to Wj+i in Wj for < j < k. Then 
V = ©jVj. Write i> = v + Ui + w 2 where Vi G Vi, i — 1, 2, and u; 2 G W 2 . As before, let A denote 
xd/dy G f). 

Lemma 7.12. Perhaps replacing v by hv for some h G if we /icrae i/ie following. 

(1) T7ie vector vq is a sum of highest weight vectors. 

(2) TTie dimension of g u (v) is one with basis A(v). 

(3) Suppose that R p C g u where p > 0. TTien /or p > i > 0, x l y p ~ % G i? p annihilates v. 

Proof. Since g u acts nontrivially on V and f is generic, there has to be a C G g u such that 
C(i>) G Wi and C(v) ^ W 2 . Then there must be a -D G t) such that D(uo + t>i) = C(f) modulo 
W 2 . Since .D preserves Vq and Vi, we must have that D{v\) = C(v ) modulo W 2 and that D 
annihilates vq. Up to the action of H, we may thus assume that vq is a sum of highest weight 
vectors or a sum of zero weight vectors. We assume the latter and derive a contradiction. Since 
g u is if-stable, we may assume that C is a weight vector for the action of C* C H. Note that D 
generates the Lie algebra of C* C H. If C has weight zero, then so does C(v) + W 2 = C(v ) + W 2 
and we cannot have that C(v) = D(v) modulo W 2 . Thus C has weight j for some j ^ so that 
C(v Q ) + W 2 = D{vi) + W 2 also has weight j. Hence v x = v[ + v" where v[ + W 2 = C(v ) + W 2 
and v[ has weight j while t>" has weight 0. Now let Z be the two-dimensional vector space 
generated by vo and v[, all modulo W 2 . The groups generated by exp(tD) and exp(tC), t G C, 
act on Z and the orbits of (wo^i) are the same. But exp(tC)(vo,v[) contains the point (vq,0) 
while exp(tD)(vo, v [) clearly does not. Hence we have (1), i.e., vq is a sum of highest weight 
vectors. Moreover, Q u (v) + W 2 is one-dimensional and generated by A(vi) + W 2 . 

Let C G g u as above. Then C(f) = D(v) for some £) G f)t, , where D is a multiple of A. 
Hence we have (2). Finally, suppose that R p C g u where p > 0. By Corollary 7.11, for i > 0, 
x % y p ~ % annihlates modulo W 2 , while ?/ p sends v to a multiple of A(v), modulo W 2 . If x % y p ~ % 
acts nontrivially on f it follows that dimg^-u) > 1. Hence we have (3). □ 

Let a : R p ®V V he the action of some R p C g u where p > 0. Let /i : V — > V be the action 
of y p via cr. We may assume that n(v) = A(v). 

Corollary 7.13. (1) For all j > 1, n j (v) = A j (v). 

(2) Ifp > 0, then for all 1 <i < p, j > 0, a{x i y p ~ l ® A j (v)) = 0. 

Proof. Suppose that p > 0. We prove (1) and (2) simultaneously by induction on j. Assume 
that ^(v) = Ai{v) for 1 < j < m and that a[x % y p ~' 1 ® A^v) = for < j < m, i > 0. We 
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certainly have the case that m — 1. Apply A to the equation a(y p <S> A m 1 (v)) = A m (v). Since 
a is equivariant, one obtains that 

Since the first term above is zero, we have that fi(A m (v)) = A rn+1 v so that, by induction, we 
have fi m+1 (v) = A m+1 (v). Now apply A to the equation a(x l y p ~ l (g) A m ~ l {v)) = 0. One obtains 
that 

a((p - i)x l+1 y p - 1 - 1 ® A m -\v)) + o{x i y v ~ i <g> A m {v)) = 
so that a{x % y p ~ % ® A m [v )) = 0. This completes the induction. In case p = 0, A commutes with 
the generator of Rq, so that (1) is immediate. □ 

Remark 7.14. Suppose that p > and that we have (1) above. Then applying A to the equations 
of (1) and using induction we obtain (2). 

Proposition 7.15. The Lie algebra g u is abelian and as H-module is multiplicity free. 

Proof. Suppose that we have copies of R p and R q in q u where we allow p = q (in which case we 
have two copies of R p ). If [R p .R q ] ^ 0, then we have a copy of some R s in g u which maps V to 
W2. Thus R s {v) 7^ while R s {v) G Wi- This implies, as in the proof of Lemma 7.12, that Q u {v) 
has dimension greater than one, a contradiction. Hence q u is abelian. If Rp has multiplicity two 
or more, then it follows from Lemma 7.12 that there is a copy of Rp which sends v to implying 
that this copy of R p acts trivially on V, a contradiction. Hence q u is multiplicity free. □ 

Proposition 7.16. For all i > 0, A l {v) is generic in Wi. 

Proof. Since v is generic in V and g u is if-stable, W\ is generated by the i7-orbit of Q u (v). 
Hence Av is generic in W\. Then the same argument shows that the i/-orbit of A 2 (v) spans 
W 2 , etc. □ 

We say that a vector w G Ri has height k if w = a^x 1 + • • • + a^x l ~ k y k where au ^ 0. A vector 
in Z := ^2 i rriiRi has height at least k (resp. height at most k) if it is generic in Z and when 
written as a sum + • ■ ■ + v^ mi where Uy is in the j'th copy of Ri, each v^j has height at 

least k (resp. at most k). 

Proposition 7.17. The H-modules Vi are multiplicity free. 

Proof. The vector A^v is generic in Wj, j > 0, and the projection of AH' to any Ri in Wj 
cannot be zero. Thus the projection of v to any Ri C Wj has height at least j. We have 
v + Wj = v o + Vi + ■ ■ ■ + + Wj where A^v G W}. It follows that A^Vi = for i < j, hence 
any v j is a sum of vectors of height at most i. Since v j G Wj it is a sum of vectors of height at 
least j. Thus A^Vj is a sum of highest weight vectors and it is generic in Wj. Hence any Ri can 
occur in Wj with multiplicity at most one. □ 

Write v = vo + ■ — h v & where Vi G V^. Then each Vi is a sum ^/gf^ v i,i where F» C N and 
lies in the copy of i?/ C V^. 

Corollary 7.18. #ac/i vector v^i has height i. 

Lemma 7.19. Let <p: i? p ® -R m — >■ Ri be equivariant and nonzero. 

(1) Necessarily m = I + p — 2i for some i with < i < min{/,p}. 

(2) Suppose that w G R m has height n < I —i. Then tp(y p <S> w) has height n + i. 

Proof. Since representations of H are self-dual, Ri appears in R p ® R m if and only if R m appears 
in R p g) Ri. Then Clebsch-Gordan implies (1). Now consider z := tp(y p ® x m ~ n y n ^ where 
m = / + p — 2i. Then Remark 7.10 shows that z ^ if the weight of y p £*D x m ~ n y n is at least — I. 
This is equivalent to n < I — i, hence we have (2). □ 
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As above, we have = J2ieFi v i,i where v^i G Ri C Vi. For any s > 0, we have W\ = 
V\ © V2 ffi • ■ ■ ffi K+i © hence we have an if-equivariant projection of W\ to V^+i. Let 

t denote a on R p <g> (Vb + ■ • • + K) followed by projection onto i?/ C V s+ \. Since we have 
cr(y p ® ^4 r (w)) = A r+1 (t;), r > 0, for every v s +i,i £ Ri C V s +i, A r+1 (v s+ ij) must be a multiple 
of r(y p g) A r (t; + ■ ■ ■ + f s )) for r > 0. Note that r vanishes on _R p ® v i)t unless t = I + p — 2j 
where < j < min{p, s}. 

Proposition 7.20. Let s and I be as above. Let Ri +P ^ 2 j C Vi, j < min-jjo, s}. If i + j > s, then 
t(R p <g> Ri+p- 2 j) = 0. 

Proof. Consider the pairs where < i' < s, < j' < min-fjo, s}, i' + j' > s and 

iV,z+p-2j' 7^ 0. Assume that i is the maximal i' that occurs and that j is the maximal j' 
that occurs in a pair {i,j')- Consider A l (vij +p -2j). It is a highest weight vector of weight 
l+p — 2j. Suppose that r{i) p ® A l (vij +p -2j)) is nonzero. Then it has height j > s — i. Moreover, 
by the choice of i and j, r(y p <g> A 1 (wj j / +p _ 2 j)) is the nonzero r(y p Cg> A l (vi t i +p -2j>)) of largest 
height (equivalently, of lowest weight). But r(y p (g> A{v)) = A t+1 {v s+ i^) where A +1 {v s+ ij) 
has height s — i. Thus r(y p £g> A l (v it i +p -2j)) must be zero. Now for < m < p we have that 
a{x m y p ~ m ® A 1 {v)) = 0. Again, by height considerations, one sees that T(x m y p ~ m ® A l {vi j +p _ 2 j)) 
must vanish. Hence t(R p ® A l {y^i +P ^2j)) = which shows that t(R p ® Ri +P _ 2 j) = 0. Now the 
proof can be completed by downward induction on i and j. □ 

For < j < minjjo, s} and Ri +P _ 2 j C Vi, the restriction of r to i? p ® Ri +p -2j is a multiple 
t itS _jT s -j of Ts-j where r s _j : R p ® Ri +P _2j — > Ri is equivariant and normalized so that T s -j{y p ® 
x l+p ~ 2 i) = x l ~ ] y° . We have that 

min{p,s} s—j 

(7.20.1) £ ^^r^V <g) A r {v lHp . 2j )) = A r+ \v s+u ) 

j=0 i=0 

for all r > where we set Vi >q = if q ^ Fj. 

Let aj and be scalars such that v s+ ij = a x l + • • • + a s+ ix l ~ s ~ 1 y s+1 and Vij +p -2j = 
bl~Q 3 x l+p ~ 2 ^ + ■ ■ • + bll 3 x l+p ~ 2 ^~ l y l whenever / + p — 2j G Fi and < j < min{p, s}. We use 
tj and bj as shorthand for i,j and ft^- respectively. For now assume that I + p — 2j G F s _j, 
j = 0,...,min{p, s}. 

Theorem 7.21. For m = 0, . . . , s } consider the equation (7.20.1) with r = m in weight l — 2s + 
2m. This gives us s + 1 equations in the unknowns t s _jb s _j for < j < min-jjo, s}. The unique 
solutions are 

ts-jbs-j = (i+p-j+i^ ( s + < j < mm{p, s}. 

Proof. First assume that p > s. Since r s _j(y p £g> x l+p ~ 2j ) = x l ~ 3 y ] \ it follows that 

r s . 3 {y p ® x l+p - 2 ^ k y k ) = -J^z^x l - j ~ k y j+k , k<l-j. 

\ k ) 

Now the 0th equation (m = 0) is 

t b + ti&i l+p lts+2 H I" ji+p-L+2jj H h tsbs Ji+p~j = ( s + 
For m = 1 the equation is 
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and the mth equation is 



m!i ro 6 m + • • • + j!/(j - ^y.tjbj +■■■+ • t a 6 a -pfr = - 

v 7— m / \s— m/ 



s — m 



Thus our system of equations is equivalent to 

V 7— m / 



j=0 v V j-m 

where Cj = tjbj/((s + l)a s+ i). Since the equations are in triangular form, there is a unique 
solution. Now the theorem will be proved if we can show that a solution to (7.21.1) is 

0© 



But one can prove this using the WZ method [WZ90]. (See [TeflO] for a brief introduction.) 
We used the implementation of the WZ method in MAPLE. 

Now suppose that p < s. We may still consider the system of equations (7.21.1). The 
solutions remain the same, but note that for j > p, the formula for c s _j gives zero. Hence the 
theorem is true even when p < s. □ 

Remark 7.22. Let < j < min{p, s}. We assumed that Ri +P _ 2 j occurred in V s -y But the 
equations force t s _jb s _j to be nonzero. Thus, in fact, Ri +P _ 2 j must occur in V s -j for there to 
be a solution of (7.20.1) for all r > 0. 

Since we are guaranteed to have vectors v s -j^ +p - 2 j in our solution of (7.20.1), what role do the 
vectors Vi t i +p -2j play for i < s—p. It is easy to see that the term involving Wj / +p _ 2 j may be elim- 
inated if we change v s - jt i +p _ 2j to v s _ jt i +p - 2 j + iff^z+ p _2j. Let us say that v" = J2j v' s _ jHp _ 2j 
is obtained from v' = J2j v s-j,i+ P -2j by an admissible modification if each v' s _^ l+p _ 2 j differs from 
v s _jj +p - 2 j by a linear combination of the Vij +p - 2 j for i < s — j. Thus we have the following 

Remark 7.23. We have a solution of (7.20.1) if and only if, up to an admissible modification of 
the v s -jj +p - 2 j, we have a solution of 

min{p,s} 

(7.23.1) t-iT-iW ® A r {v s ^ l+p - 23 )) = A r+1 {v s+lJ ), r > 0. 

5=0 

Proposition 7.24. Let v s+ ij G V s+ i and v s _j t i +p ^ 2 j G V s -j have coefficients dj and b s ~^ as 
above. Fix the b s _j, < j < minjjo, s}. Then there are unique values of the t s _j and b s s jj m for 
m < s — j such that there is a solution of (7.23.1). 

Proof. We know that the t s -j are uniquely determined. We only need to show that the b s s jj jm 
for m < s — j are unique satisfying (7.23.1). This is easy because of the triangular form of 
the equations. For r = 0, the equation in weight I reads t s b s s = a±. For arbitrary r < s, the 
equation in weight / is r\t s b s s r = (r + l)!a r+ i. For r = s this is one of the equations we considered 
in Theorem 7.21 and we have b s s r = ^c^+i for r < s. Now suppose that we have determined 
the b s s Zg j for < q < m. Consider (7.23.1) in weight I — 2m with r = 0. It gives an expression 
for t s - m b s s Z™fi in terms of the a» and b s s , s ,_^ for s' > s — m. Thus we may solve for b s s Z™,o- F° r 
< r < s — m we obtain an equation that we can solve for b s s Zmr- The equation that we get 
for 6 s _ m is one of the equations that we considered in Theorem 7.21. Hence given ax, ... , a s+ i 
and the b s _j, there are unique t s _j and b 8 s Zj m solving (7.23.1). □ 
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Remark 7.25. Suppose that I + p — 2j £ Fi for alH + j < s, < j < min{p, s}. Then we may 
modify the v s _jj +p -2j admissibly so that the b s s Zj m , m < s — j , are arbitrary. Hence there are 
t s _j giving solutions of (7.23.1) (after admissible modifications) and giving solutions of (7.20.1) 
(without changing any vectors). 

Let us formulate the conditions that need to be satisfied to have R p C Q u , p > 0. 

Definition 7.26. Let v £ V be generic. We say that v satisfies (* p ) if 

(1) We have a decomposition V = ©f =0 where the Vi are multiplicity free if-modules. Let 
Fi C N such that Vi = ® ieFi Ri. 

(2) Possibly replacing v by hv for some h £ H, we have that v = £\ ^2 leF . v^i where 
Vij £ Ri C Vi has height i. 

(3) For every v s+ i t i £ K+i, s > 0, we have that / +p — 2j £ F s _j for < j < min{p, s}. Let 
the t s _j be given by Theorem 7.21. Then the vectors v s -jj +p -2j, perhaps after admissible 
modification, are solutions of (7.23.1). 

Theorem 7.27. Let v £ V be generic. Then R p C Q u , p > 0, if and only if v satisfies (* p ). 

Proof. We have shown that R p C g u implies that (* p ) holds. Conversely, if (* p ) holds, then we 
have constants t i)S _j such that (7.20.1) is satisfied. Let r denote the corresponding map 

min{p,s} s—j 

R p®( RHv-2 i cV i )^R l cV a+ x. 

j=0 i=0 l+p-2j£Fi 

The various mappings r combine to give us an equivariant map a: R P ®V — > V . It follows from 
Remark 7.14 that o{x l y v ~ % (g) v) = for i > 0. By construction, a(R p w) is one-dimensional 
and generated by cx(|/ p <S> v) = A{v). If S denotes the copy of Ftp C End(^) corresponding to o", 
then [S 1 , S](v) = which implies that [S, S] acts trivially on V, i.e., [S, S] = 0. By construction, 
S consists of nilpotent transformations. Now by Proposition 7.6 we have S C g u . □ 

Corollary 7.28. Suppose that there is a generic v £ V such that q u is not zero or the trivial 
H -module. Then there are subsets F , . . . , Ff. C N such that V = ®*L ©z Sj f. Ri an d p > such 
that for every I £ F s+ i, s > we have I + p — 2j £ F s _j for < j < min-Jjo, s}. 

Corollary 7.29. The group G normalizes H if V does not satisfy the condition of Corollary 
7.28. In particular, G normalizes H in the following cases. 

(1) V is an isotypic H-module. 

(2) The multiplicity of Ri is at least two, where I is maximal such that Ri C V. 

Proof. Part (1) is clear. In case (2), there has to be a vector v s+ ij where s > 0. Thus we must 
have R p +i C V s , which obviously fails. □ 

Using Remark 7.25 it is clear that one can have extremely complicated situations where 
R P C Q u . Here is a modestly complicated case. 

Example 7.30. Let V = Ri +p -2 © Ri+2 P , V\ = Ri +P and V 2 = Ri, where I > 1, p > 0. Let 
v = vo t i +p - 2 + vo,i+2 P + v i t i +p + v 2 ,i £ V = V © Vi © V 2 where the v TiS are of height r in R s C 
Then by Remark 7.25, g M contains a copy of R p . Here we have that a{y p © f o,z+2 P ) = ^( v i,i+p) 
and cr(|/ p © A r (voj +p -2 + ^l^+p)) = ^4 r+1 ('W2,/), t = 0, 1. If we add a copy of to V\ and a copy 
of Ri to Vo (assume p ^ 2) with corresponding components i^z and wo,« in v, then we also have 
a copy of i?o m Qu- If P — 2, we already have Ri C Vq and we only have to add Ri C Vi and Vij. 

Example 7.31. Suppose that V = 2Ri © Ri-i © -R/+i where / > 2. Then it is possible to have a 
generic v £ V such that i?! C0 U . However, one can check that this is not possible if we increase 
the multiplicity of Ri to 3. 
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8. Appendix 

Here we establish the branching rules which are used in Table 2 and the calculation of V K in 
cases 6.3 and 6.4. Recall that if <p is a G-module, then S(ip) = Q)k<S k ((p) where <S fc (y?) denotes the 
subspace of S k (ip) obtained using Cartan multiplication of the irreducible subrepresentations of 
if. 

Let n = 2m + 1, m > 1. Let X denote the cone in </? n _i(D n ) which is the closure of the orbit 
of a highest weight vector. Consider the action of SL„ xC* on 0(X) where <pi(D n ) = C 2n = 
<Pi(SL n ) ©y? n _i(SL n ) = C n © (C n )*. Here C n is the span of the positive weight vectors of D n , C* 
acts on C n with weight 2 and on (C n )* with weight —2. As SL n xC* representation, tp n _i(D n ) 

is ^ n ffiy2 n _ 2 (SL n ) ©z/ n _4 + y2„_ 4 (SL n ) ©z/ n „ 8 H h<^i(SL n) © v_ n A r 2 and ip n (D n ) = </?n-l(D 

y?„_i(SL n ) © ^„_ 2 H h ^(SL 

— n- 

Theorem 8.1. LetX be the closure of the highest weight orbit in (p n _i(D n ). Then, as (SL n xC*)- 
module, 0(X) = S(<p n (D n )). 

Proof. It is well-known that X is normal, and every point of X except the origin is smooth. For 
x G u n , x 7^ 0, x is a smooth point of X, x is a fixed point of SL n and the slice representation 
of SL n at x is 6i + y? n -2- Since v 9 n-2 has no invariants, (9(X) SL ™ is generated by a coordinate 
function z on Then z is not a zero divisor in O(X), hence 0(X) is a free C[z]-module. By 
Luna's slice theorem [Lun73], O(X) is a free C[z]-module on 0(<£> n _ 2 ). But 0((p n -2) = S 1 ^) 
is just the sum of all representations of the form (p^ 2 • V 9 ^ 4 • • • V^n-i 1 eac h with multiplicity one. 
It follows that the products of the highest weight vectors of the restriction of y9 n _i(D n )* to SL n 
freely generate the highest weights of 0(X) as an SL n -module and as an (SL n xC*)-module. □ 

Now suppose the n = 2m, m > 2. Let X denote the closure of the orbit of a highest 
weight vector of (p n -i(D n ). Consider the action of SL n xC* C D n such that <pi(D n ) becomes 
C rt ©z/i©(C n )*®^_i. Effectively, we have the action of GL n . Then <£> n _i(D n ), as a GL n - module, 

is V m + (y9 n _2 © Z/ m -2 H V V- m - 

Theorem 8.2. Let n = 2m, m > 2 and let X be the closure of the highest weight orbit in 
y9 n _i(D n ). Then, as GL n -module, O(X) = «S(<^ n _i(D n )). 

Proof. Let z± be coordinate functions on the copies of v± m in <p n _i(D n ). As above, one computes 
that there is a slice representation (y?2m-2 + #i>SL 2m ) for the action of SL 2m on X. The slice 
representation has a quotient of dimension two and principal isotropy group C m . It follows that 
the GL n -invariants have dimension 1, hence they must be generated by z + Z-. Moreover, the 
only way that the trivial SL n - representation can occur in C[<£> n „2 ® v-m-2 + ■ • • + <^2 © 
is in products whose C*-weight is a multiple of ±m (just count boxes in Young diagrams). 
Since GL n is spherical in D n , each i/f. m , k G Z, occurs once in the free C [z+ Z-]- module 0(X). 
Thus the SL n -invariants must be the polynomial ring C[z + , zJ\ and 0(X) is free over C[z + , z_}. 
For the corresponding map X — > C 2 , the general fiber is SL n /C m , which gives that the only 
SL n -representations that occur are y^ 2 . . . <y2n-~2 2 for 02, ... , a n _2 > 0, each with multiplicity one. 
It follows that 0(X) = S(<f n ^x(D n )). □ 

Finally, we consider the case where X is the closure of the highest weight vector in (p n (D n ), 
n = 2m > 4. As GL n -module, we have ip n {D n ) = <p n -i © v m -\ © ■ ■ • © <fi © ^-m+i- 

Theorem 8.3. As GL n -module, 0(X) = S(ip n (D n )). 

Proof. There are no invariants in this case, so we have to proceed a little differently. We first find 
a general point of A. Let ei, . . . , e n be the usual basis of C n . Let u = C2 Ae3 + - • • + e2 n -2 A e2 n -i 
considered as an element of the Lie algebra of D n . Then the action of exp(w) on e\ sends it to 
the sum v of the elements e\ Au k G <f2k+ij fc = 0, . . . , m— 1. The isotropy group H of SL n acting 
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on v is the semidirect product of C m _i with Hom(C • e n , C™" 1 ) © Hom(C n " 2 , C • ei) where C n ~ 2 
here stands for the span of e 2 , . . . , & n -\ an d C n_1 stands for C n ~ 2 © C • e±. Note that our copy of 
C m _i acts standardly on C n ~ 2 . Now dim SL n /H = dimX, so that SL n -v is a dense orbit in X. 
Since X is factorial [VP72, Theorem 4], any divisor in the complement of the dense orbit must 
be defined by a semi- invariant of SL n , hence by an invariant. Thus there are no such divisors, 
so that the complement of SL n -v has codimension 2. It follows that 0(X) ~ 0(SL n /H). But 
the irreducibles of SL n with an if-fixed vector are those of the form fl 1 ^ 3 ■ ■ ■ f^Si where the 
dj are nonnegative, and the fixed point set has dimension one. Thus 0(X) is as claimed. □ 

We now compute the ring of i^-invariants in the cases (6.3) and (6.4) of Table 2. 

Proposition 8.4. Let X (resp. Y) be the closure of the orbit of the highest weight vector of 
(p 7 (D 8 ) (resp. <p 8 (D 8 )). Consider the action o/B 4 on X andY where <^i(D 8 )|B 4 = <^ 4 (B 4 ). Then 
0(X) B ± = C[/ 4 ] and 0(Yf± = C[/ 2 ,/ 3 ] where deg/, = i. 

Proof. Using LiE [vLCL92, vL94] one computes that the Poincare series of 0{X) B " i is l + t 4 + . . . 

and that the Poincare series of 0{Y) Ba is 1 + t 2 + t 3 + t 4 + t 5 + Recall that X and Y are 

normal, hence so are 0(X) Bi and 0(Y) Bi . Thus dim 0(Y) Bi > 2. The restriction of ip 7 (D 8 ) 
(resp. ^ g (D g )) to B 4 is </?i<^ 4 (resp. (pf+{p 3 ). Let P (resp. Q) be the stabilizer of the highest weight 
line in ip 7 (D 8 ) (resp. <p 8 (D 8 )). Then the Levi components L(P) and L(Q) of P and Q double 
cover representatives of the two SOi6-conjugacy classes of embeddings of GL 8 in SOi6- We have 
L(P) ~ (SL 8 xC*)/(Z/4Z) and the same for L(Q). Restricted to L(P), ^ 7 (D 8 ) becomes the 
representation z/ 4 + A 6 (C 8 )®^ 2 + A 4 (C 8 ) + A 2 (C 8 )®z/_ 2 + z/_ 4 . The highest weight space of tp 7 (D 8 ) 
is v^. The tangent space to X at a nonzero point of 1/4 is z/ 4 + A 6 (C 8 ) £g> vi so that dimX = 29. 
The restriction of ^ 7 (D 8 ) to L(Q) is A 7 (C 8 ) ® u 3 + A 5 (C 8 ) (8) v x + A 3 (C 8 ) © v_ x + C 8 © i/_ 3 . For 
ip 8 (D 8 ), the decompositions relative to L(P) and L(Q) are reversed, so dimY = 29, also. 

Consider the action of H = AdSL 3 on C 9 as v 9 iV 9 2 + @\- Then ^4(64) \H = 2</? 1 <^ 2 . Clearly 
the image of H in SOi 6 lies in a copy of GL 8 . Suppose that this copy of GL 8 is double covered 
by a conjugate of L(P). Then X H 7^ (0), and the B 4 -orbit of a nonzero fixed point is closed 
since the normalizer of H in B 4 is a finite extension of H [Lun75, 3.1 Corollary 1]. It is easy 
to check that the isotropy group of a nonzero point of X H is at most a finite extension of H. 
Thus the dimension of the corresponding closed B 4 -orbit is 28. Hence dimO(X) B4 < 1 and the 
Poincare series information gives that 0(X) Bi = C[/ 4 ] where deg/ 4 = 4. If our copy of GL 8 
were double covered by a conjugate of L(Q), then we would see that dimO(Y) B4 < 1, which is 
a contradiction. Thus 0(X) Bi is as claimed. 

Now consider the group K = S0 6 x S0 3 C S0 9 . Then the double cover K of K is (SL 4 x SL 2 ) / ± 
/ and y? 4 (B 4 ), as if-representation, is C 4 Cg> C 2 + (C 4 )* ® C 2 . Thus K is a subgroup of a copy 
of GL 8 in SOi6- If this GL 8 is double covered by a conjugate of L(P), then one sees that there 
are no nonzero fixed points of K (actually K) in </? 8 (D 8 ). But </> 8 (D 8 )|B 4 = ip\ + ip 3 has infixed 
points of dimension 2. Hence our copy of GL 8 is double covered by a conjugate of L(Q) and 
the weight space 1/4 of the restriction of <y? 8 (D 8 ) to L(Q) lies in Y and is fixed by K. The 
group Nq a (K) / K ~ Z/2 flips the highest and lowest weight spaces z/± 4 . Since K is a maximal 
connected reductive subgroup of B 4 , the stabilizer of v± is K and any point of z/ 4 lies on a closed 
orbit. The slice representation of K is S' 2 (C 4 ) + 9\ which shows that the principal isotropy 
group H of the action of B 4 (actually SO9) is S0 3 x S0 3 x S0 3 . It follows that dim Y"/B 4 = 2. 
Now Nso g (H)/H ~ jy(D 3 ), the Weyl group of D 3 , where V := ip 8 (D 8 ) H has dimension 5. 
One easily computes that the generators of 0{V) W ^ D3S) are of degree at most 5. Then by the 
Luna-Richardson theorem [Lun75, 3.2 Corollary] it follows that the invariants of 0(Y) S ° 9 have 
generators in degree at most 5, and then from our information about the Poincare series it 
follows that C(F) S ° 9 = C[/ 2 , fs) where degfc = i, i = 2, 3. □ 
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Remark 8.5. There is no representation of SOg with principal isotropy group H = SO3 x SO3 x SO3 
and slice representation S' 2 (C 4 ) + Q\ of K = S0 6 x S0 3 which has homogeneous invariants / 2 
and f'3 of degrees 2 and 3, respectively. The reason is that we would have a slice which is an 
open i\~-invariant subset of the linear subspace V — C ■ v + S 2 (C A ) where K fixes v, and the 
restrictions of the /j to V would have to be functions of C ■ v alone since the invariant of S 2 (C A ) 
is of degree 4. Thus f% and fs would be algebraically dependent, a contradiction to normality. 

Remark 8.6. The generators fa and ^3 form a homogeneous regular sequence in 0(Y), hence 
0(Y) is a free graded C[/2, /sj-module [Sta79, Lemma 3.3]. It follows that 0(Y) is cofree, i.e., 
each module of covariants is free over C[f 2 , fs\- Of course, we have the analogous result for 
0{X). 



References 



[Dyn52] E. B. Dynkin, Maximal subgroups of the classical groups, Trudy Moskov. Mat. Obsc. 1 (1952), 39-166, 
AMS Translations, Vol. 6, 245-378. 

[GL97] Robert M. Guralnick and Chi-Kwong Li, Invertible preservers and algebraic groups. III. Preservers 
of unitary similarity (congruence) invariants and overgroups of some unitary subgroups, Linear and 
Multilinear Algebra 43 (1997), no. 1-3, 257-282. 

[G093] V. V. Gorbatsevich and A. L. Onishchik, Lie transformation groups, Lie groups and Lie algebras, I, 
Encyclopaedia Math. Sci., vol. 20, Springer, Berlin, 1993, pp. 95-235. 

[Gur94] Robert M. Guralnick, Invertible preservers and algebraic groups, Proceedings of the 3rd ILAS Con- 
ference (Pensacola, FL, 1993), vol. 212/213, 1994, pp. 249-257. 

[Gur97] , Invertible preservers and algebraic groups. II. Preservers of similarity invariants and over- 
groups o/PSL„(F), Linear and Multilinear Algebra 43 (1997), no. 1-3, 221-255. 

[Hcl78] Sigurdur Hclgason, Differential geometry, Lie groups, and symmetric spaces, Pure and Applied Math- 
ematics, vol. 80, Academic Press Inc. [Harcourt Brace Jovanovich Publishers], New York, 1978. 

[Hoc65] G. Hochschild, The structure of Lie groups, Holdcn-Day Inc., San Francisco, 1965. 

[Jac62a] N. Jacobson, A note on automorphisms of Lie algebras, Pacific J. Math. 12 (1962), 303-315. 

[Jac62b] Nathan Jacobson, Lie algebras, Intcrscience Tracts in Pure and Applied Mathematics, No. 10, Inter- 
science Publishers (a division of John Wiley & Sons), New York-London, 1962. 

[Jac79] , Lie algebras, Dover Publications Inc., New York, 1979, Republication of the 1962 original. 

[DL94] Dragomir Z. Dokovic and Chi-Kwong Li, Overgroups of some classical linear groups with applications 
to linear preserver problems, Linear Algebra Appl. 197/198 (1994), 31-61, Second Conference of the 
International Linear Algebra Society (ILAS) (Lisbon, 1992). 

[LP01] Chi-Kwong Li and Stephen Pierce, Linear preserver problems, Amer. Math. Monthly 108 (2001), 
no. 7, 591-605. 

[Lun73] D. Luna, Slices Stales, Sur les groupes algebriques, Soc. Math. France, Paris, 1973, pp. 81-105. Bull. 

Soc. Math. France, Paris, Mcmoire 33. 
[Lun75] , Adherences d'orbite et invariants, Invent. Math. 29 (1975), 231-238. 

[Oni62] A. L. Oniscik, Inclusion relations between transitive compact transformation groups, Trudy Moskov. 

Mat. Obsc. 11 (1962), 199-242, AMS Translations 50 (1966), 5-58. 
[Oni69] , Decompositions of reductive Lie groups, Mat. Sb. (N.S.) 80 (122) (1969), 553-599, Math. 

USSR-Sb. 9 (1969), 515-554. 
[Oni94] Arkadi L. Onishchik, Topology of transitive transformation groups, Johann Ambrosius Barth Verlag 

GmbH, Leipzig, 1994. 

[DP93] Dragomir Z. Dokovic and Vladimir P. Platonov, Algebraic groups and linear preserver problems, C. 

R. Acad. Sci. Paris Ser. I Math. 317 (1993), no. 10, 925-930. 
[PD95] Vladimir P. Platonov and Dragomir Z. Dokovic, Linear preserver problems and algebraic groups, Math. 

Ann. 303 (1995), no. 1, 165-184. 
[Rai'07] Mustapha Rai's, Notes sur la notion d'invariant caracteristique, arxiv.org/abs/0707.0782vl. 
[SchOl] Gerald W. Schwarz, Algebraic quotients of compact group actions, J. Algebra 244 (2001), no. 2, 

365-378. 

[Sta79] Richard P. Stanley, Invariants of finite groups and their applications to combinatorics, Bull. Amer. 

Math. Soc. (N.S.) 1 (1979), no. 3, 475-511. 
[TeflO] Akalu Tefera, What is a Wilf-Zeilberger pair?, Notices Amer. Math. Soc. 57 (2010), no. 4, 508-509. 



LINEAR MAPS PRESERVING ORBITS 



27 



[vL94] M. A. A. van Leeuwen, HE, a software package for Lie group computations, Euromath Bull. 1 (1994), 
no. 2, 83-94. 

[vLCL92] M. A. A. van Leeuwen, A. M. Cohen, and B. Lisser, A package for Lie group computations, CAN 

(Computer Algebra Nederland) , Amsterdam, 1992. 
[VP72] E. B. Vinberg and V. L. Popov, A certain class of quasihomogeneous affine varieties, Izv. Akad. Nauk 

SSSR Ser. Mat. 36 (1972), 749-764, English translation Math. USSR Izv. 6 (1972), 743-758. 
[Wey39] Hermann Weyl, The Classical Groups. Their Invariants and Representations, Princeton University 

Press, Princeton, N.J., 1939. 
[WZ90] Herbert S. Wilf and Doron Zeilberger, Rational functions certify combinatorial identities, J. Amer. 

Math. Soc. 3 (1990), no. 1, 147-158. 



Department of Mathematics, Brandeis University, Waltham, MA 02454-9110 
E-mail address: schwarz@braoideis.edu 



